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^  The  purpose  of  this  research  was  to  address  the  problem  of 

facilitating  group  decision  performance  in  a  dynamic  task  Situation 

which  involves  uncer tainty  through  the  use  of  graphic  information 

presentation  and  decision  heuristics.  The  investigation  involved  a 

laboratory  experiment  with  groups  of  three  performing  a  dynamic  group 

choice  task  (the  Team  Resource  Allocation  Task).  Each  team  completed  a 

total  of  32  trials,  half  of  which  were  presented  at  a  fast  rate  of  speed 

and  half  at  a  more  moderate  speed.  The  teams  had  insufficient  resources 

to  respond  to  every  event  presented,  so  they  were  advised  to  identify 

and  commit  resources  to  the  most  valuable  combinations  of  events. 

Visual  coding  schemes  for  presenting  the  events  varied  between  teams  as 

did  the  presence  or  absence  of  decision  heuristics  provided  by  the 

researcher. 

* 

The  most  important  finding  of  this  research  was  that  aids  which 
guided  the  decision  process  had  a  much  greater  impact  on  decision 
quality  in  a  dynamic  group  choice  task  than  did  the  form  in  which 
information  was  presented.  This  finding  suggests  that  how  decision 
makers  use  technology  (their  decision  process)  is  at  least  as  important 
as  the  technology  itself  for  supporting  decision  making.  Additional 
research  should  be  applied  to  exploring  how  to  aid  the  decision  process. 

Other  findings  Included  the  following:  (1)  Teams  performed  better 
under  moderate  time  pressure  than  under  high  time  pressure,  (2)  Teams 
performed  better  with  practice,  (3)  Heuristics  had  a  greater  effect  on 
lerision  quality  under  moderate  time  stress  than  under  high  time  stress, 
and  (4)  Heuristics  had  an  immediate  and  lasting  effect  on  decision 
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1.0.  INTRODUCTION 


In  spite  of  the  myriad  of  studies  of  group  performance  completed, 
'wo  still  know  vary  littla  about  why  some  groups  art  mora  affactiva  than 
others.  We  know  even  lass  about  what  to  do  to  improve  the  performance 
of  a  given  group  working  on  a  specific  task.'  t HACKB3 ,  p.  331] 
Computer ■ based  support  of  group  decision  making  and  the  explicit 
discussion  of  decision  strategies  have  both  been  offered  as  means  to 
improve  group  performance.  This  researoh  examines  how  group  decision 
making  performance  is  affected  by  the  provision  of  a  Group  Deoision 
Support  System  (GDSS) .  In  this  study  two  types  of  deoision  support  are 
provided  to  decision  making  groups,  one  type  involving  the  information 
presented  for  decision  making  and  ths  other  type  representing  an  aid  to 
the  process  used  by  the  group  to  make  a  decision.  The  goal  of  the 
research  is  to  investigate  the  role  that  may  ba  played  by  each  of  these 
types  of  deoision  support  on  group  deoision  making  performance. 

This  opening  ahapter  identifies  the  general  problem  addreeeed  by 
the  researoh,  details  the  purpose  of  the  study,  and  explains  the 
relevance  of  the  reeearch  in  general  and  of  the  chosen  experimental  task 
in  particular.  The  final  four  sections  of  the  chapter  introduce  the 
research  approach,  present  the  limitations  and  ksy  assumptions  of  the 
study,  Identify  the  contributions  and  strengths  of  ths  research,  and 
outline  the  contents  of  the  remaining  ohapters  of  the  thesis. 

1 . 1  The  Problem 

Researah  in  decision  making  has  followed  a  number  of  different 

avenues,  some  more  developed  than  others.  There  are  ample  opportunities 

among  the  less  developed  areas  for  further  research.  For  example; 

Research  in  the  computer-based  support  of  decision  making 
groups  is  only  beginning  [GALLB5],  Additional  research-based 
knowledge  about  decision  proctlies  is  essential  for  the 
development  of  more  effective  Group  Decision  Support  Systems 
(GDSS);  at  tho  same  time,  the  development  and  application  of 
GDSS  tools  as  experlmsntal  treatments  can  contribute  greatly 
to  studying  group  deoision  process* s  [KRAE84 ] . 

The  number  of  variablee  involved  in  and  the  complexity  of 
group  deoision  making  situations  make  studying  these 
situations  difficult,  so  empirically  based  knowledge  about 
group  deoision  making  procsssss  is  limited  [KRAE64 ] ,  Among 
the  variables  that  Hackman  and  Morris  present  as  controlling 


factors  In  group  performance  are  the  task  performance 
strategies  used  by  group  members  [HACK83],  There  is  much  to 
be  discovered  empirically  about  how  groups  might  use 
heuristics  as  part  of  a  Strategy  to  support  their  decision 
making  processes. 

There  is  a  growing  body  of  researoh  concerning  the  impact  of 
computer-based  business  graphics  on  decision  making.  The 
results  of  graphics  research,  however,  have  been  inconsistent 
[DESA84,  JARV05],  The  inconsistent  results  have  been 
attributed  largely  to  task  differences  CDICK80,  JARV80a], 
Research  is  still  needed  to  identify  appropriate  matches 
between  the  decision  maker's  task  and  the  means  of  presenting 
information  to  the  decision  maker. 


Laboratory  studies  that  have  attempted  to  explain  Improved 
decision  outcomes  associated  with  computer-based  decision 
support  have  been  especially  limited.  The  missing 
explanations  can  be  discovered  by  examining  how  the  decision 
making  proaess  changes  with  variations  in  the  decision  support 
provided  [BENB841. 

No  single  study  oan  fill  all  the  gaps  in  even  one  of  these  areas, 
but  any  study  of  decision  making  should  be  aimed  at  contributing  some 
knowledge  to  one  or  more  of  the  areas.  The  following  purpose  statement 
identifies  the  gaps  which  are  targeted  in  this  research. 

1.2  Research  ffyiESfi 

The  purpose  of  this  researoh  is  to  explore  the  impact  of  variations 
in  information  presentation  and  of  decision  process  interventions  on 
group  decision  quality.  In  particular,  this  research  addresses  the 
problem  of  facilitating  Aroup  performance  in  a  dynamic  task  situation 
which  involves  uncertainty  through  the  use  of  groohlc  information 
presentation  and  decision  heuristics. 

1 . '•!>'  Relevance  of  the  Research 

Nearly  every  facet  of  our  society  --  political,  legislative, 
judicial,  economic  --  functions  through  decisions  made  by 
groups..,.  The  functioning  of  every  business,  educational, 
and  political  organisation  relies  on  decisions  made  by 
management  teams  [FISH80,  p.  2], 

Group  decision  making  is  indeed  pervasive  in  our  society,  and  Huber 
t HUBEQ4* ]  predicts  that  group  decision  making  in  the  future  will  need  to 
be  oven  more  frequent  and  faster  and  will  need  to  account  for  greater 
complexity  in  the  decision  situation.  He  expects  that  organisations 
will  improve  the  effectiveness  of  their  decision  making  groups  through 
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the  adoption  of  more  sophisticated  group  technology ,  The  more  that 
can  bo  learned  about  the  impact  of  group  technologies  on  decision  making 
teams,  the  more  organisations  will  be  prepared  to  cope  effectively  with 
the  demands  of  an  information-intensive  future. 

This  study  extends  knowledge  of  the  effects  on  group  decision 
outcomes  of  two  factors:  computer-based  graphic  information 
presentation  and  the  explicit  provision  and  discussion  of  decision 
heuristics.  In  particular,  the  study  extends  knowledge  about  the 
effects  of  both  factors  in  a  dynamic  dscislon  environment,  an 
environment  that  has  not  bsen  extansivsiy  studied.  In  addition,  a  major 
contribution  of  this  rsssaroh  is  the  opportunity  to  explain  fcow 
variations  in  decision  support  contributes  to  the  effectiveness  of 
decision  making  groups, 

1.4  Relsvanos  of  tha  Experiments!  Ink 

The  Team  Resource  Allocation  Problem  (TRAP),  the  experimental  task 
for  this  research,  places  the  decision  making  team  in  a  decision 
environment  that  has  not  btsn  extensively  Studied,  but  that  is 
characteristic  of  the  post-industrial  Society  envisioned  by  Huber 
[!KU)E84b] .  Results  of  the  rsssaroh  are  applicable  in  any  organisation 
in  which  teams  of  decision  makers  must  make  choices  under  time  pressure 
in  dynamic,  uncertain  situations. 

for  example,  among  a  computer  operations  team's  basic  duties  is 
responding  to  system  messages  displaysd  on  ths  computer  console.  A 
number  of  messages  may  be  displayed  simultaneously  on  the  same  console, 
and  the  situation  is  constantly  changing.  The  operations  team  has  to 
choose  from  among  ths  messages  presented  the  on#  that  requires  ths 
highest  priority  rssponss.  Becauss  of  the  high  turnover  and  the 
resulting  low  experience  level  among  computer  operators,  organisations 
are  interested  in  facilitating  operator  training  and  task  performance, 
Sperry  Corporation,  in  particular,  is  sponsoring  research  aimed  at 
improving  operator  task  performance  through  improvements  in  the 
computer-oparator  intorface  [VANW881.  The  results  of  this  research  are 
applicable  in  the  development  of  such  user  interfaces  and  in  the 
development  of  training  programs. 
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Public  and  private  crialfl  intervention  teams  could  alio  benefit 

from  using  systems  that  incorporate  research  findings  on  supporting  team 

decision  making.  Steven  Pink,  President  of  Lexicon  Communications 

Corporation,  a  management  consulting  firm  specializing  in  crisis 

management ,  insists  that  ‘Every  crisis  demands  a  crisis  management  team* 

and  identifies  the  need  to  collect  the  appropriate  Information  and  route 

it  to  the  proper  decision  makers  [F1NKBG,  pp.  57,  65}.  Crisis  teams 

require  quick,  reliable,  and  dear  information  to  support  their 

decisions  at  times  when  stress  and  time  pressure  can  lead  to  information 

overload  and  miscommunioation  (H0US861.  An  information  system  designed 

to  support  crisis  management  must  'minimize  message  ambiguity, 

Information  filtering  and  distortion,  and  conflicting  instructions' 

(H0USQ6,  p.  3931.  The  results  of  this  study  aid  in  understanding  how  to 

minimize  ambiguity,  filtering,  distortion,  and  conflicting  instructions 

in  crisis  management  situations  through  expliolt  discussions  of  decision 

strategies  and  appropriately  designed  information  displays, 

Additional  spooifio  applications  of  the  research  results  involve 

decision  making  teams  in  the  following  decision  environments: 

Manufacturing/prooess  control ,  Teams  must  monitor  ongoing 
operations,  identifying  and  responding  to  problem  situations, 

Air  traffic  control/military  command  and  aontrol.  Information 
in  these  control  situations  'usually  ooncerns  the  past, 
present,  and  future  location,  identity  and  certain  other 
attributes  of  various  objects'  to  which  teams  must  respond 
quickly  and  accurately  [WOHL84], 

Financial  markets.  The  Stock  markst  is  volatile,  and 

financial  managers  must  react  quickly  to  new  quotations  for 
the  benefit  of  their  olients. 

In  each  of  these  environments,  team  decision  making  can  potsntially  bt 
made  more  effective  through  use  of  decision  support  which  incorporates 
the  results  of  this  research. 

1 . 5  Research  Approach 

The  study  was  a  laboratory  sxperimsnt,  which  providsd  the  means  for 
controlled  tests  of  the  effects  of  Specific  independent  variables.  As 
recommended  by  Benbasat,  the  experiment  was  conducted  with  multiple 
dependent  variables  and  using  multiple  methods  of  measurement  [ BENB84  3 . 
The  dependent  variables  measured  group  performance  and  perceptions  of 
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workload.  The  measurement  method*  used  wara  computer  tracking  and 
questionnaire*.  Thaia  method*  allowed  the  analyaia  of  both  the  daciaion 
outcome*  (what  affect  the  independent  variable*  had  on  decision 
outcome*)  and  the  daciaion  proa***  (how  and  why,  the  independent 
variable*  had  an  influence). 

1.6  Limitation*  and  Key  Assumption* 

1.6.1  Limitation* .  The  study  i*  limited  in  the  following  way*: 

The  study  i*  specifically  oriented  to  exploring  the 
performance  of  three-person  team*  with  the  TRAP.  The  result* 
may  not  be  general  liable  to  other  group  size*  or  to  other 
task* . 

Only  a  limited  set  of  the  variables  involved  in  the  group 
proce**  were  examined.,  Praotical  limitations  demand  that  many 
factor*  be  reserved  fdr  other  studies. 

Only  a  limited  number  of  possible  treatments  within  each  of 
the  variables  were  examined.  Different  information 

presentation  approaches.  Set*  of  heuristics,  or  levels  of  time 
pressure  could  yield  different  results,  but  practical 
limitations  again  demand  that  many  variations  be  reserved  for 
other  studies. 

1.C.2  Assumptions .  The  study  was  based  on  the  following  key 
assumptions : 

'  College  students  are  suitable  subjeots  for  the  research.  The 
characteristics  of  the  oollege  students  who  served  as  subjects 
for  the  experiment  may  not  exactly  match  those  of  the 
individuals  who  would  ultimately  use  systems  based  on  the 
results  of  the  study.  Especially  for  exploratory  research 
[D1CK86],  however,  the  use  of  students  as  surrogates  for 
decision  makers  should  not  discredit  the  study  [REMU861. 

Factors  such  as  the  mix  of  men  and  women  on  a  team,  the 
seating  position  of  a  mala  on  a  mostly  female  team  (or  of  a 
female  on  a  mostly  male  team),  and  the  mix  of  levels  of 
individual  experience  with  personal  computers/computer 
terminals  and  video  game*  on  a  team  would  not  materially 
affect  the  results  or  would  be  sufficiently  accounted  for 
through  random  assignment  of  treatments.  This  assumption  was 
supported  in  preliminary  analysis  of  the  experimental  results. 

Discriminating  among  the  five  levels  in  the  information  coding 
schemes  would  remain  within  the  limits  of  human  information 
processing  CMZLL66 } .  The  subjects  were  tested  to  assure  that 
they  could  discriminate  among  and  understand  the  meanings  of 
the  five  levels  in  the  code  they  used,  but  the  test  was 
administered  under  static  conditions  with  no  time  pressure. 
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Tilt  assumption  was  that  the  dynamic  condition*  and  the  tima 
pressure  of  tha  experimental  situation  did  not  placa  tha  codta 
bayond  tha  llmita  of  human  information  processing. 

1 .7  Contribution*  of  tha  Raaaarch 

A  major  contribution  of  tha  study  la  that  it  axttnd*  knowledge 

about  information  rapraaantation  and  hauriatlca  af facta  in  an  Important 

decision  oontaxt  that  ha*  not  batn  cxtanilvaly  atuditd.  In  describing 

tha  likaly  nature  of  post-industrial  organization* ,  Huber  predict*  that 

organisational  daclaion  making  will  be  more  complex;  concurrently,  there 

will  be  demand*  for  organisational  decision  making  to  ba  both  more 

frequent  and  faster.  To  achieve  the  requisite  decision  group  efficiency 

and  sf f activenesa .  organisations  will  be  motivated  to  integrate  computer 

and  communication  technology  into  their  deoision  process**,  especially 

for  the  high  stress  conditions  facing  crisis  deoision  groups  [HUBEB4b], 

This  raaaarch  contributes  to  determining  the  most  appropriate 

Applications  of  information  technology  to  group  decision  making  in  post** 

industrial  organizations. 

Other  major  strengths  of  the  study  are  as  follows; 

It  is  founded  in  established  reference  discipline  theory 
[KEEN80  ,  JARV8B ,  DESA84] . 

It  builds  on  both  the  managerial  graphics  research  stream  at 
the  University  of  Minnesota  and  the  team  technology  researoh 
stream  it  the  Harry  Q.  Armstrong  Aerospace  Medical  Researoh 
Laboratory.  The  Sharing  of  definitions,  concepts,  and 
measurement  Instruments  helps  to  build  a  cumulative  tradition 
of  MIS  research  [KEEN0O ,  D1CXB0 ] . 

Tha  equipment  used  represents  state-of-the-art  technology  in 
graphics  terminals,  yet  the  study  fooused  on  the  af f eotiveness 
of  tha  support  provided  to  decision  makers  rather  than  on  the 
technology  itself  IKEEN80] . 

The  results  of  the  study  have  practical  application  both  to 
the  sponsoring  organisation  and  to  other  organizations 
involved  in  related  tasks  tKEENBO], 

Tha  dynamic  group  task  used  simulates  a  decision  context  that 
has  not  been  extensively  studied. 

The  study  provided  opportunity  for  a  limited  examination  of 
the  cognitive  processes  underlying  the  decisions  mad* 
IPAYN78) . 
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1 . 0  Organization  of  the  Thesis 

Chapter  2  review*  significant  prior  research.  Chapter  3  present*  a 
conceptual  model  placing  the  Study  in  context)  identified  the  variable! 
of  lnteroat,  and  develop*  hypoth****  from  the  relevant  theory  and 
rooearoh  re*ult*,  Chapter  4  provide*  a  detailed  explanation  of  the 
experimental  task  and  a  full  deaoription  of  the  reaearoh  method,  The 
|  research  outcome*  are  presented  in  Chapter*  5,  Analysis  of  Results,  and 

0,  Discussion  of  Result*  and  Conclusion*. 
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3,0  SIGNIFICANT  PRIOR  RESEARCH 


The  specific  purpose  of  this  research  is  to  address  the  problem  of 
facilitating  <roup  decision  performance  in  a  dynamic  task  situation 
which  involves  uncertainty  through  the  use  of  iraohlc  Information 
prerontation  and  decision  heuristics.  The  following  sections  present 
highlights  of  prior  research  relevant  to  the  areas  Indicated  by  each  of 
thn  underlined  phrases  in  the  purpose  statement. 

2 . 1  Group  Decision  Making 

In  a  review  of  group  researah  contributions  over  a  period  of  three 
years  from  the  disciplines  of  psychology,  sociology,  communications, 
education,  and  administrative  sciences,  McGrath  and  Kravitc  determined 
Umt  about  40  percent  of  the  studies  were  concerned  with  the 
effectiveness  of  group  task  performance  CMGGR823,  The  large  proportion 
of  performance-oriented  studies  continued  an  historical  tendency  to 
focus  a  major  share  of  research  effort  on  suoh  Issues  as  comparing  group 
to  individual  performance  and  identifying  factors  that  affect  group  task 
performance,  Among  the  tasks  most  frequently  used  were  those  identified 
in  McGrath's  Task  Ciroumplex  as  ohoics  tasks,  both  intellectivs  and 
deals  ion- making  tasks  tMCGR843, 

Group  decisions  are  different  from  those  of  Individuals.  In  spits 
of  the  potential  problems  in  rsaching  consensus  and  the  opportunity  for 
conflict,  a  group  tends  to  make  decisions  of  higher  quality  than  the 
average  performance  of  individual  members  of  the  group  [MINE84,  MCGRB41, 
Group  decision  making,  however,  also  tends  to  be  inefficient  and  slow; 
groups)  tend  to  proceed  in  bursts  of  activity,  jumping  from  one  issue  to 
another  frequently,  while  individuals  Sssm  able  to  sustain  conosntration 
on  «  single  issue  CFISH9Q], 

2.1.1  Process  Gains  and  Losses 

llubtr  suggests  that  a  group's  actual  decision-making  of f ectiveness 
equals  its  potential  ef f ectiveness  plus  gains  resulting  from  group 
processes  minus  losses  resulting  from  group  processss  [HUBE82],  Groups 
gain  because  they  have  more  resources,  mors  sources  of  Information  and 
of  new  ideas,  and  more  perspectives  for  critiaal  analysis  of  Ideas 
IF1SHB0].  They  generate  more  alternatives  and  are  better  at  analyzing 


the  relative  advantage*  and  diiadvantage*  of  the  varloue  alternative* 
[TIJR082  ] . 

On  the  other  hand,  group  prooe****  can  contribute  to  lo**e«  in 
decision  ef fectivene** .  For  example,  deciiion  quality  can  suffer  when 

*  i 

individual  group  member*  dominate  the  group  prooe**  beyond  the  merit*  of 
their  contributions,  when  group  member*  miacommunicate ,  or  when  there 
are  group  pressure*  to  conform  [HUBE621,  The  complexity  of  group 
communication  limit*  the  group's  ef fectivene**  especially  when  group 
productivity  depend*  on  the  coordinated  effort*  of  group  member* 
UIACK83).  Group*  also  fail  to  reach  their  potential  beaause  they  are 
ineffective  at  identifying  member*  who  could  contribute  most  to  the 
group  decision,  For  example,  the  quality  of  a  group  decision  ha*  been 
shown  to  be  no  different  from  the  quality  of  the  individual  decision 
sulucUd  a*  best  by  member*  of  the  group,  By  contrast,  the  group 
decision  was  inferior  to  the  individual  deoiaion  that  aatually  was  the 
best  of  the  decision*  mad*  by  member*  of  the  group  t Ml NE64 ] . 

3.1.2  Task  Effect*  on  Group  Performance 

Group  decision  processes  vary  according  to  the  task.  Intellective 
tasks  such  a*  the  TRAP  task  used  in  this  study  have  a  oorrect  answer, 
For  such  tasks,  the  decision  scheme  that  best  fits  group  performance  ha* 
been  labelled  ‘truth,  supported,  wins*  CM0GR82],  In  other  words,  the 
group  accepts  a  solution  as  oorreot  if  at  least  two  members  know  it  to 
be  correct.  A  solution  presented  by  only  one  member,  even  if  it  is  the 
correct  solution,  may  not  be  readily  accepted  unless  the  solution  can  be 
explained  easily  and  is  intuitively  compelling  once  revealed,  Groups 
accept  solutions  to  'eureka'  problems,  then,  on  a  'truth  wins'  basis 
( MOORS 2 ] .  Other  factors  associated  with  the  task  (e.g,,  time  pressure) 
also  influence  group  performance  C I SENS  1 ] .  Specifically,  Isenberg 

found  that  increased  time  pressure  led  to  less  equal  sharing  of 
communication  time  among  group  members,  more  salient  leadership,  and 
less  accurate  decisions. 

2.1.3  Computer-based  Support  of  Group  Decision  Making 

Examining  the  impact  of  computer-based  support  of  group  decision 
making  is  a  relatively  new  area  of  study.  A  Group  Decision  Support 
System  (GDSS)  has  been  defined  as  ‘an  interactive  computer-based  System 
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which  facilitates  solution  of  unstructured  problems  by  a  set  of  decision 
makers  working  together  as  a  group*  [DESA85].  QDSS  results,  although 
limited,  provide  some  basis  for  understanding  the  impact  of  computer- 
based  support  of  group  decision  making. 

Only  a  small  number  of  QDSS1  have  been  developed  [HUBE84a],  Of 
those  dosigned,  most  have  been  intended  to  facilitate  deoision  processes 
that  fit  the  rational  model  by  helping  to  Identify  and  clarify  decision 
alternatives  IKRAE84],  Any  one  system,  however,  tends  to  support  only  a 
small  set  of  spscifia  tasks  [HUBE84*], 

Steeb  and  Johnston  [STEE81]  compared  the  performance  of  groups  with 
and  without  computer-based  aids  in  Solving  a  complex  international 
crisis  problem.  The  system  elioited  a  decision  tree  from  the  group 
baaed  on  pooled  inputs  from  the  members,  Qroups  with  computer  support 
considered  more  factors,  scored  better  on  measures  of  the  content  and 
breadth  of  their  solutions,  and  developed  more  detailed  courses  of 
action.  They  were  also  more  confident  in  their  decisions  and  more 
satisfied  with  the  process,  Qroups  without  oomputer  support  had  the 
advantage  only  in  completing  the  process  in  less  tlms, 

Turoff  and  Hilts  CTUROB33  alio  compared  the  performance  of  groups 
with  and  without  computer-based  support,  The  task  was  a  semistruotured 
survival  problam,  and  the  computer-based  support  Involved  computer 
conferencing  and  a  decision  aid  in  the  form  of  feedback  concerning  the 
degree  of  consensus  achieved.  Communications  in  groups  operating  with 
computer  conferencing  facilities  were  more  task-oriented  than  in  groups 
operating  face-to-face.  Second,  groups  with  either  the  decision  aid  or 
formal  leadership  reached  greater  consensus  In  computerised  conferences, 
but  groups  with  both  the  decision  aid  and  formal  leadership  reaohed  no 
greater  consensus  than  groups  with  neither  the  decision  aid  nor  formal 
londnrship. 

i 

Group  Decision  Support  System  use  has  also  bsen  shown  to  enhance 
decision  quality  for  groups  performing  a  problem-solving  task  [ LEWIS 2 ] , 
a  problem- i inding  task  [GALL881,  and  idea  generation  in  support  of  an 
organisational  planning  task  UPPL88J,  Another  Study  provided  partial 
support  for  ths  contention  that  use  of  a  QDSS  provides  for  more  even 
distribution  of  influence  behavior  among  group  members  performing  an 
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intellective  task  [ZIGU88].  On  th«  other  hand,  GDSS  use  did  not 
increase  group  consensus  on  a  preference  allocation  task  [WATS87 ] . 

Nearly  all  the  studies  of  group  decision  performance  have  used 
static  decision  tasks;  i.e.,  the  task  environment  remains  stable  during 

!  the  decision  process,  The  next  sub-section  reviews  some  studies  of 

decision  making  in  dynamic  task  environments. 

2.2  Dynamic  Decision  Waking  Tasks 

Rapoport  suggested  that  a  decision  making  situation  aan  be 
described  as  dynamic  when  molt  or  all  of  the  following  characteristics 
are  involved:  (1)  decision  makers  make  a  series  of  decisions  over  time; 
(2)  previous  decisions  or  other  factors  may  change  task  requirements 
over  time;  (3)  the  results  of  previous  decisions  may  determine  what 
information  is  available  for  further  decisions;  (4)  the  results  of  any 
decision  may  impact  future  events  CRAP079]. 

In  spite  of  the  fact  that  real  decisions  are  made  primarily  in 
dynamic  rather  than  static  situations  ILITTBO],  research  in  dynamic 
decision  making  has  been  extremely  limited.  One  reason  for  the  limited 
research  has  been  that  dynamic  decision  models  ‘are  so  complex  end 
require  So  many  assumptions  that  the  Interpretation  of  experimental 
results  is  typically  ambiguous'  C SL0V77 ,  p,  14],  In  addition,  many 
studies  have  been  plagued  by  the  'ourse  of  ineensitivlty'  CRAP07B] .  In 
other  words,  when  decision  makers  are  told  to  optimise  their  performance 
and  are  compared  on  the  basis  of  payoffs  received,  large  variations  in 
decision  behavior  typically  result  in  very  small  differences  in  payoff, 
On  the  other  hand,  Slovic,  Fischhoff  and  Lichtenstein  suggest  that  there 
are  some  important  human  factors  questions  worth  studying  in  the  area, 
including  'How  do  variations  in  the  basic  system  (e.g.  different 
instructions  or  information  displays)  affect  people's  performance? ‘ 
[f:'.LiUV77  ,  p,  201 . 

2,2.1  Elon-Slmul lentous  Choice 

Some  studies  of  dynamic  decision  making  have  explicitly  established 
decision  points  at  which  dsoislon  makers  had  the  option  either  to  make  a 
final  decision  immediately  or  to  seek  edditional  alternatives  before 
making  a  final  choice.  The  point  at  which  a  final  decision  is  made 
provided  Some  evidence,  for  example,  about  whether  the  decision  makers 
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hnve  adopted  a  satisficing  or  a  maximizing  strategy.  Qlander  found  that 
&  maximizing  strategy  best  described  individual  behavior.  There  were, 
however,  some  individuals  who  tended  toward  satisficing  when  they  had 
very  little  information  about  the  alternatives  that  would  bo  available 
with  search  [0LAN78). 

'A  .'A,  2  Supervisory  Control 

•  Other  studies  have  ooncentrated  on  tasks  related  to  supervisory 
control  of  unstable  systems,  such  as  those  faced  by  air  traffic 
controllers,  manufacturing  prooess  controllers,  and  power  system 
regulators.  The  human  operator's  task  in  these  problems  is  to  monitor 
various  sources  of  information  and  to  Select  and  act  on  those  events 
that  demand  their  intervention.  Such  studies  often  compare  decision 
maker  performance  to  a  model  of  idealised  performance.  Although  further 
work  is  called  for  on  the  effects  of  uncertainty  in  such  tasks,  there 
have  been  some  consistent  findings,  Many  deoision  makers  performed  very 
well,  performance  improved  rapidly  with  practice,  and  some  subjects 
either  consistently  under-oontrolled  or  over-oontrol led  the  system 
IRAP0781. 

Pattipati,  Kleinman,  and  Ephrath  C  PATTQ3  3  also  focussed  on 
supervisory  control  problems  using  a  dynamic  deoision  task  adapted  from 
a  task  designed  by  Tulga  and  Sheridan  [TULQBO ] .  The  Pattipati  et  al, 
task  presented  up  to  five  events  at  a  time  on  a  cathode  ray  tube  (CRT) 
screen.  Each  event  was  represented  as  a  rectangle  that  appeared 
Initially  at  the  left  edge  of  the  soreen,  moved  across  the  screen  to  the 
right  at  various  speeds  on  ont  of  five  lines,  and  disappeared  upon 
reaching  the  right  edge  of  the  screen.  No  more  than  one  rectangle  at  a 
time  appeared  on  any  one  line.  The  height  of  each  rectangle  represented 
its  payoff  and  could  be  one,  two,  or  throe  units.  The  number  of  dots 
displayed  on  the  rectangle  represented  the  number  of  seconds  required  to 
process  the  event  and  could  range  from  one  to  five  seconds.  Subjects 
processed  an  event  by  pressing  a  button  corresponding  to  the  line  on 
which  the  rectangle  appeared.  Subjects  were  told  to  maximize  their 
accumulated  payoff  and  were  told  their  scores  at  the  end  of  each  90- 
second  trial.  Pattipati,  Kleinman,  and  Ephrath  were  not  directly 
interested  in  comparing  task  performance  among  individuals,  so  they  did 
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not  report  performance  data.  Instead,  they  developed  a  Dynamic  Decision 
Model  (DDM)  to  describe  human  performance  on  the  experimental  task. 

The  DDM  was  based  on  the  assumption  that,  subject  to  human 
limitations,  well-trained  participants  would  behave  rationally  (would 
use  subjectively  expected  value  to  choose  among  available  rectangles). 
The  model  took  Into  account  such  factors  as  the  processing  time  required 
(or  an  event,  the  time  available  to  work  on  the  event,  the  relative 
Importance  (reward  value)  of  the  event,  human  reaction  delays,  and  human 
inconsistencies  in  responding  to  similar  events,  Empirical  studies 
showed  that  their  model  more  accurately  predicted  human  performance  than 
did  model',  based  on  task  scheduling  theory  [PATT831. 

2.2.3  Team  Resource  Allocation  Problem 

Brown  and  Leupp  [BROW05]  extended  the  experimental  task  developed 
by  t'attipati,  Kleinman,  and  Ephrath  in  developing  the  Team  Resource 
Allocation  Problem  (TRAP).  The  major  extension  involved  adapting  the 
task  for  use  with  groups  of  three  decision  makers  who  were  expected  to 
coordinate  their  responses  to  the  task.  All  the  events  in  a  TRAF  trial 
moved  at  the  same  speed  and  required  the  same  processing  time,  but 
trials  could  run  at  different  Speeds  with  event  proaessing  time  adjusted 
relative  to  the  trial  speed.  TRAP  presented  a  maximum  of  11  events  on 
the  screen  at  a  time  and  expanded  the  range  of  event  values.  (See 
Chapter  5  for  a  full  description  of  the  TRAP.) 

In  a  baseline  study  using  the  TRAP,  Brown  and  Leupp  determined  that 
performance  of  three-person  teams  was  not  strongly  affected  when  the 
TRAP  was  presented  on  one  large  screen  display  rather  than  three 
individual  CRT's.  Although  the  specific  results  concerning  large  screen 
technology  was  not  a  concern  in  this  research,  the  Brown  and  Leupp  study 
did  establish  &  research  methodology  on  which  additional  studies  could 
be  built  [BR0W85].  Continued  use  of  variations  on  the  TRAP  throughout  a 
senes  of  experiments,  with  retention  of  the  underlying  measuring  tools, 
overcomes  the  difficulties  associated  with  a  proliferation  of  measuring 
instruments  as  identified  by  Jarvenpaa  et  al.  C J ARV85  3 . 

The  environment  for  decision  making  in  real  organisations  is  not 
only  dynamic;  it  is  also  uncertain.  The  next  sub-section  outlines  what 
research  has  shown  concerning  decision  making  under  uncertainty. 
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2 . 3  Decision  Making  Under  Unaertalnt 


A  state  of  uncertainty  exists  when  a  decision  maker  has  incomplete 
information  on  which  to  base  the  decision.  Decisions  under  uncertainty 
have  often  been  subcategorised  as  either  decisions  under  risk  or 
decisions  under  uncertainty.  Decisions  under  risk  involve  knowledge  of 
all  the  possible  decision  outcomes  and  their  probability  distribution; 
decisions  under  uncertainty  involve  knowledge  of  all  the  possible 
decision  outcomes  but  not  their  probability  distribution  (MUHS8U. 

2 .  3 .  1  Expected  Utility  Theory 

The  dominant  theory  under  which  study  of  decision  making  under 
uncertainty  has  been  conducted  is  the  iubjeotive  expected  utility  model 
( SL0V77 ] ,  It  has  buen  used  as  both  a  normative  and  a  descriptive  model 
of  human  risky  choice  behavior  t KAHN79 ) .  According  to  the  model, 

'rational'  managers  weight  the  utilities  of  outaomes  by  their 
probabilities  and  choose  among  alternatives  on  the  basis  of  the  highest 
expected  utility  [ MUHSB 1 ] .  Expected  utility  theory,  however,  has 
frequently  been  criticised  as  inadequate  to  desoribe  human  behavior. 
People  are,  for  example,  systematically  biased  in  their  perception  of 
uncertainty  [QRET70]. 

2.3.2  Human  Biases 

Human  biases  result  from  applying  msntal  operations  that  simplify 
judgments  but  that  also  lead  to  errors.  Tversky  and  Kahneman  C TVER7 4 ] 
have  identified  three  such  mental  operations  (heuristics); 
representativeness,  availability,  and  adjustment  and  anchoring,  The 
first  heuristic,  representativeness,  can  lead  to  errors  through 
insufficient  attention  to  base  rates,  sample  sixes,  or  the  reliability 
of  evidence.  The  availability  heuristic  biases  judgments  of 
probabilities  according  to  how  readily  examples  come  to  mind.  Finally, 
people  tend  to  'anchor'  on  and  then  to  make  adjustments  from  some 
starting  probability  estimate,  but  the  adjustments  tend  to  be  smaller 
than  the  evidence  warrants.  Therefore,  different  ’anchors'  yield 
different  probability  estimates. 

2.3. 3  Prospect  Theory 

In  addition  to  being  biased  in  their  Judgments  of  uncertainty, 
humans  are  often  inconsistent  in  their  choices.  They  frequently  do  not 


use  obvious  optimal  strategies  for  risky  choice  problems  t  L I TT80  3 .  To 
Account  for  human  inconsistencies (  Kshnemsn  end  fvtrsky  t K AHN70 ]  have 
proposed  prospect  theory  as  an  alternative  to  expected  utility  theory. 
A  prospect  is  one  of  the  'gambles'  or  probabilistic  options  from  which 
the  decision  maker  is  to  choose. 

According  to  prospect  theory  [KAHN79],  people  go  through  two  phases 
in  choosing  among  prospects:  an  editing  phase  and  then  an  evaluation 
phase.  In  the  editing  phase,  people  simplify  the  statement  of  the 
prospects  through  such  operations  as  ooding  (formulating  the  prospect  in 
terms  of  gains  and  losses  from  the  ourrent  position),  rounding  off 
probabilities  and  outcomes,  and  discarding  extremely  unlikely  outcomes, 
In  the  evaluation  phaae,  the  value  of  each  outcome  is  weighted,  not  by 
the  probabilities  of  the  outaomes,  but  by  a  decision  weight  that  tends 
to  overweight  very  low  probabilities  and  to  underweight  all  other 
probabilities. 

The  combined  effeots  of  the  editing  and  evaluation  phases  account 
for  many  of  the  inconsistencies  humans  exhibit  in  risky  choices.  For 
example,  prospect  theory  provide!  an  explanation  for  why  humans  tend  to 
select  certain  gains  over  merely  probable  gains  with  comparable  or  even 
higher  expected  value  and  to  select  probable  losses  over  certain  losses 
l  KA11H79  ] . 

2.3.4  Alternative  Theories 

Additional  theories  have  been  offered  as  alternatives  to  prospect 
theory  for  explaining  human  risky  choice  behavior.  For  example,  regret 
an'1  disappointment  theories  eaoh  suggest  that  the  feelings  one 
anticipates  having  upon  resolution  of  the  gamble  are  factors  in 
evaluating  the  alternatives  CL00M93,  BELL82 ,  BELL63 ,  LOOM06L  Regret  is 
the  feeling  an  individual  has  when  it  appears,  after  the  fact,  that  he 
ha?  made  the  wrong  choice  (the  horse  he  almost  played  wins), 
disappointment  rosults  when  outcomes  fail  to  match  up  to  expectations 
(his  number  wins  the  lowest  prise  in  a  lottery) .  According  to  regret 
and  disappointment  theory,  then,  human  inconaistencies  in  risky  choice 
behavior  can  be  explained  as  rational  tradeoffs  between  objective 
utility  and  avoidance  of  or  insurance  against  anticipated  regret  or 
d I snppointment , 
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2 .  .1 ,  S  Risky  Ohoiot  Behavior  in  Qrouos 

Some  early  researoh  concerning  riaky  choice  behavior  in  groups 
suggested  that  groupa  tended  to  take  greater  riaka  than  the  group 
numbers  would  aa  individuals,  a  finding  labelled  the  ‘risky  shiit* 
(FH5H80] .  Subsequent  studies.  however,  failed  to  support  the 
genoraliaability  of  the  risky  shift,  Instead,  it  seems  that  groups  may 
shift  (‘choice  shift*  or  'group  shift') .  but  not  always  in  the  risky 
direction;  factors  such  as  the  significance  of  the  choice  outcomes 
apparently  influence  the  direction  and  magnitude  of  the  shift  [MUHSBll. 

The  difficulties  of  making  decisions  under  dynamic,  uncertain 
conditions  call  for  aids  for  the  decision  maker.  Among  the  alternative 
decision  aids  are  computer-based  graphics  and  decision  heuristics,  which 
will  be  addressed  in  the  following  two  sub-sections, 

2 , 4  Oraphlo  Information  Presentation  fcr  Decision  Maklns 

In  reporting  on  research  opportunities  in  the  decision  and 
management  sciences  as  identified  by  the  National  Science  Foundation. 
Little  CLITT66]  included  the  need  to  study  the  effectiveness  of 
alternative  computer-based  forms  of  presenting  information.  Ho  and 
others  have  noted  especially  that  Stata-of-tha-art  computer  graphios 
technology  provides  the  means  and  the  motivation  for  rasaarch  in  the  use 
of  oolor  and  graphics  (LITTSO ,  JARVBOb .  DAVIB3 ] . 

2.4.1  Tables  versus  Graphs 

The  body  of  research  in  the  use  of  business  graphios  has,  in  fact, 
been  growing,  Early  studies  fooused  largely  on  comparing  the 
of f wctiveness  of  tabular  and  graphic  presentations  of  information  for 
decision  support  with  mixtd  rasults  [IVES82,  DESA84),  Among  the  reasons 
cited  for  the  conflicting  results  are  the  following  [BENBB0B,  JARVB9); 

a.  Inappropriate  experimental  designs 

b.  Differences  in  types  of  graphics  used  (bar  charts,  pie  charts, 
etc , ) 

c.  Lack  of  either  e  theoretical  or  an  empirical  basis  for 
hypotheses 


d.  Task  differences 

Improved  technology,  used  aooording  to  established  guidelines  for 
preparing  graphics  (IVES02,  0LEV85],  can  now  provide  high  quality 


graphic*  lor  lair  aompariaon*  with  w*l 1-prepared  tabular  format*.  In 
addition,  reaearohera  hava  atartad  to  account  explicitly  for  thair 
research  baaaa  and  lor  the  alia eta  ol  task  dillaranca*  in  thair  reaulta. 
2.4.2  Raaaaroh  ProSram* 

Raaaarohara  hava  alao  atraaaad  the  importance  of  studying  the  uae 
of  graphic*  in  program*  ol  reaaaroh  rather  than  in  one-shot  experiment* 
[BENBBOc,  D1CKB6],  Dickson  at  al,  reported  on  a  aerie*  ol 
experiments  uaing  three  diflerant  taaka;  the  relative  ellactivaneia  ol 
graphical  and  tabular  lormata  varied  with  the  demand*  of  the  aaaigned 
task.  They  recommended  further  inveitigationa  of  talk  content,  taak 
complexity,  and  taak  atructure  a*  dimension*  ol  the  taak  environment  a* 
it  affect*  the  u*c  ol  graphic*  (D1CK88J.  Benbaaat  at  al.  alao  conducted 
a  aerie*  of  three  experiment*,  but  uaing  a  single  resource  allocation 
task;  their  result*  reinforced  the  concept  that  information  presentation 
needs  to  be  matched  with  the  demand*  ol  the,  task,  In  their  experiment*, 
graphical  lormata  allowed  faster  deoiaion  making  and  were  superior  tor 
identifying  pattern*  in  the  data:  tabular  format*  allowed  higher  quality 
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duc<*ions  and  were  superior  lor  identifying  speoifio  data  value* 
[ BENB05 ,  BENBQOa ,  BENBBOb,  BENBBOo] ,  In  a  program  ol  researoh  uaing  the 
Team  Resource  Allocation  Problem  (TRAP),  deoiaion  performance  uaing  a 
graphic  repreaentation  ol  the  taak  wa*  found  to  be  superior  to  decision 
performance  uaing  a  tabular  repreaentation  ol  the  taak,  especially  under 
high  time  stress  condition*  [WIL8871,  In  that  the  TRAP  is  essentially  a 
monitoring  application  in  which  the  data  presented  is  inherently 
dynamic,  these  results  support  Davis  and  Sweaty's  findings  that  oomputer 
graphics  are  particularly  appropriate  lor  dynamic  data  presentation 
ID AVI  831. 

Since  a  graphic  representation  has  been  established  as  superior  bo 
a  tabular  representation  lor  presenting  information  in  a  dynamic  group 
decision  task  (TRAP),  comparisons  among  alternative  graphic 
representations  are  appropriate.  Graphic  representation*  ol  information 
can  vary  along  a  variety  ol  dimensions.  For  example,  Ives  [IVES82] 
identified  live  basic  visual  input  channels  available  to  the  human 
information  processing  system:  oolor,  relative  position,  brightness, 


movement,  and  shape.  Appropriate  us*  of  th*s*  channels  has  bt«n  shown 
to  aid  in  target  identification,  a  fundamental  aspect  of  the  TRAP. 

2.4.3  Feature  Integration  Theory 

Treisman  and  Golada's  Feature  Integration  Theory  suggests  that  for 
tasks  sucli  as  visual  search  and  target  Identification  and  localization, 
visual  cues  that  are  conjunctions  of  features  (e.g.,  color  and  shape) 
require  focused  attention  and  seem  to  be  processed  serially.  Cues 
having  only  a  single  feature  (e.g.,  oolor  alone)  may  be  processed  in 
parallel  ( 1 . e . ,  more  quickly).  Further,  when  multiple  conjunctively 
coded  objects  are  presented  under  time  constraints  or  when  attention  may 
be  diverted,  an  individual  may  incorrectly  combine  features  of 
unattended  objects  and  falsely  report  the  presence  of  specified  objeots 
(illusory  conjunctions).  Thus,  especially  under  conditions  when 
attention  may  be  diverted  or  overloaded,  subjects  perform  visual  search 
tasks  more  quickly  and  accurately  when  the  objects  vary  on  a  Single 
dimunsion  rather  than  as  conjunctions  of  features  tTREISO,  TREI82], 

Because  other  researchers  had  found  qualitative  changes  in 
performance  on  visual  tasks  with  extended  practiac,  Treisman  and  Qelade 
examined  the  effeats  of  practice  on  searching  for  conjunctively  coded 
objects.  They  found  no  indication  of  movement  from  serial  to  parallel 
processing  over  13  blocks  of  practice  ITREIBO], 

Others  havs  tempered  Treisman  and  Qelads's  findings.  Egeth  et  al. 
suggest  that  search  for  conjunctively  coded  objects  may  proceed  with  an 
initial  parallel  elimination  of  all  objects  lacking  ths  more  salient 
specified  feature.  A  second  Stage  in  the  search  involves  serial 
examination  of  the  remaining  objeuts  for  the  specified  conjunction  of 
features  (EQET843 .  For  example,  a  search  for  red-A  objeots  may  start  by 
eliminating  all  objects  that  are  not  red  and  then  searching  serially  lor 
A' s  among  the  red  objects.  Sagi  and  Julies  agreed  that  eingle-fsetured 
objects  are  deteoted  in  parallel,  but  they  found  that  precise 
identification  of  a  detseted  objeot  requires  focal  attention  on  that 
object  [SAQI891.  Performance  differences,  however,  depend  more  on  the 
time  required  to  locate  the  object  than  on  the  time  required  to  identify 
the  value  of  the  objeot  (0HR183). 
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2.4.4  Color  versus  Letters 

Considering  only  single-featured  visual  codes,  both  alphanumerios 
and  color  hr.ve  bean  evaluated  as  excellent  (or  use  in  locating  objeota, 
although  alphanumeric*  are  auperior  (or  preolae  identK ioation  o(  the 
object  t DA V 1831 ,  Research  haa  alio  shown  that  more  diatlnctive  colon 
and  shape*  allow  (eater  aearoh  than  leia  diatlnctive  onea;  i.e,,  aearch 
is  (aster  (or  distinguishing  red  (rom  green  or  P  (rom  Q  than  (or 
distinguishing  blue  (rom  green  or  0  (rom  Q  CTBE180 ,  QHAN62). 

A  considerable  body  of  research  exists  concerning  the  relative 
effectiveness  oi  viaual  codes  (or  search  and  identification  tasks, 
particularly  comparing  color  to  other  codes.  Christ  reviewed  42  studies 
published  between  1952  and  1073  on  the  e((eota  of  color  on  visual  search 
and  identification  performance.  In  ten  studies  on  the  acouracy  of 
identifying  objects  in  a  unidimensional  display,  letters  were  superior 
to  color,  and  the  advantage  (or  letters  increased  with  increases  in  the 
density  of  the  display  and  with  decreases  in  exposure  time.  On  the 
other  hand,  in  (our  studies  on  the  time  required  to  locate  objects, 
color  was  superior  to  letters,  although  the  difference  was  less  than 
that  found  between  color  and  other  visual  codes  (site,  brightness, 
geometric  shapes,  etc.)  [CHRI73], 

In  a  more  recent  set  of  experiments,  Christ  determined  that  there 
are  'no  clear  and  consistent  advantages  (or  any  one  visual  code  set  over 
the  others*  CCHRI 03 ,  p.  93)  Where  he  found  differences  in  the  relative 
uf f octi veness  of  the  codes,  they  depended  on  other  display  conditions 
(e.g,,  density  o(  the  display),  the  task,  and  the  dependent  measure 
used.  Differences  did  exist  under  those  conditions  most  closely  related 
to  the  conditions  established  for  this  study;  for  locating  a  specified 
object  among  12  randomly  located  objeots  in  a  display,  location  times 
were  shorter  for  colored  dots  than  for  letters  (GHRI831,  Christ  has 
also  demonstrated  that  extended  practice  with  a  visual  task  tends  to 
attenuate  any  differences  in  performance  based  on  use  of  different 
single -featured  codes  (letters,  digits,  familiar  geometric  shapes,  and 
colored  dots)  (CHRI75,  CHRI833. 

Although  additional  work  is  required  to  clarify  how  bast  to  match 
visual  codes  with  task  conditions,  appropriate  graphic  displays  of  task 
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information  could  contribute  to  decision  making  effectiveness.  The 
decision  maker's  decision  process,  which  is  also  an  important  variable 
contributing  to  decision  making  effectiveness,  may  be  influenced  by 
decision  heuristics.  Studies  about  the  impact  of  decision  hourlstios  on 
decision  making  will  be  addressed  in  the  following  sub-section. 

2  . 0  Heuristics  for  Decision  MaklnS 

A  number  of  studies  have  described  the  heuristios  and  biases  that 
humans  typically  use  in  a  variety  of  task  situations  [HUBEQ5;  SL0V771. 
Much  has  been  written,  for  example,  about  human  biases  in  situations 
involving  uncertainty  and  our  failure  as  intuitive  statisticians. 
Humans  seek  to  reduce  the  complexity  in  the  problems  they  face  through 
the  use  of  heuristios,  which  reduce  the  number  of  alternatives 
considered  to  a  manageable  else  EFULL7B3. 

The  heuristics  often  work  well  but  may  lead  to  serious  errors 
tKLI£lB2b].  For  example,  human  preference  for  simplioity  led  to 
subcptimali by  in  a  study  exploring  people's  aholces  between  conflicting 
analyse*  of  a  decision  problem  [KLEI82al.  Klein  found  that  people  were 
morn  confident  in  their  decisions  with  a  simple  Intuitive  analysis  than 
with  a  conflicting  but  mathematically  accurate  analysis,  He  called  for 
additional  research  on  training  deoision  makers  bo  recognise  their 
biases  and  to  know  <«h*n  to  legitimately  use  heuristics. 

(liven  that  the  heuristics  that  decision  makers  adopt  are  sometimes 
biased,  some  effort  has  been  given  to  examining  the  impact  of  attempts 
to  reduce  the  biases.  Procedures  for  reduoing  human  bias  are 
collectively  known  as  debiasing  methods  C F 1 SC62 ] .  The  nature  of  a 
particular  debiasing  method  depends  on  the  perceived  source  of  the  bias: 
if  the  bias  results  from  a  faulty  task,  fix  the  task;  if  the  bias 
results  from  a  biased  but  perfectible  decision  maker,  provide 
tralning/feedback  CFIS062 3 ,  Assuming  a  perfectible  deoision  maker  in  a 
probabilistic  task,  training  could  take  the  form  of  providing 
prescriptive  decision  rules,  or  alternate  heuristios,  with  the  intent  of 
determining  their  impact  on  decision  performance. 

Cats-Baril  and  Huber  examined  the  impact  of  providing  decision-aid 
heuristics  on  the  performance  of  en  ill-structured  career  planning  task. 
The  task  required  participants  to  identify  career  objectives,  generate 
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al Urnative  strategies  lor  achieving  the  objectives,  and  prioritizing 
tluv  al ternativea .  Some  participants  were  provided  with  a  purpoae- 
expansion  heuriatic  designed  to  inoreaae  the  number  of  objectives  and 
alternatives  considered  and  to  explors  the  alternatives  more  thoroughly. 
Participants  with  heuristics  performed  better  both  on  objective  measures 
of  the  number  usues  addressed  and  on  expert  evaluations  of  the 
quality  of  the  pl^ns  produced.  The  researchers  called  lor  additional 
studios  on  the  impact  of  different  types  of  heuristics  on  tasks  with 
different  levels  of  Structure  tOATS87), 

Johnson  and  Payne  have  defined  risky  choioe  heuristics  as  rules 
that  systematically  simplify  the  choice  among  alternatives  by 
disregarding  some  elements  of  the  problem  s pece  (ignoring  some 
alternatives.  Selectively  examining  outcomes,  ignoring  some  event 
Information,  eta.),  Different  heuristics  are  based  on  different 
simplifications  of  the  choios.  Using  computer  simulation,  they  compared 
six  houristios  under  a  variety  of  conditions  lor  thsir  accuracy 
(conformity  to  a  choice  using  expected  value)  and  mental  effort 
required .  Their  data  suggested  that  'heuristics,  in  at  laast  some  task 
environments,  can  approximate  the  accuracy  of  normative  rules  with 
substantial  savings  in  sffort*  [J0HN85,  p.  408). 

Kleinmunts  C KLBt 83 ]  also  ussd  oomputar  simulation  to  study  a 
variety  of  heuristics  for  medical  diagnosis,  He  determined  that  the 
relative  performance  of  the  various  heuristics  depended  more  on  time 
pressure  and  feedback  on  the  outcome  of  previous  treatments  than  on 
factors  such  as  diataoe  bale  rates.  Klelnmuntz  concludes  in  part  that 
additional  research  is  needed  on  how  task  knowledgs  is  acquired  and  how 
heuristlas  are  selected. 

Arkee  et  el.  studied  conditions  under  which  individuals  would 
choose  not  to  use  helpful  but  Imperfect  decision  heuristics  in  e 
probabilistic  task.  In  one  experiment,  subjects  were  told  (correctly) 
that  using  the  heuristics  provided  would  result  in  70  percent  accuracy 
in  their  Judgments.  Those  who  were  warned  that  deviating  from  the 
heuristic  would  result  in  degraded  performance  and  who  received  no 
monetary  incentives  for  performance  did  best,  matching  the  70  percent 
accuracy  level,  Those  who  were  encouraged  to  try  to  improve  on  the 


heuristic  or  who  received  monetary  incentives  for  performance  deviated 
from  the  heuristic  and  judged  less  accurately.  Those  who  received  no 
immediate  feedback  on  the  accuracy  of  their  judgments  also  outperformed 
those  who  did  receive  feedback;  feedbaak  about  an  incorrect  judgment 
tended  to  cause  deviation  from  the  heuristic  and  degraded  performance  on 
tho  following  trial,  In  a  second  experiment,  the  heuristic  provided 
resulted  in  75  percent  aocuracy.  Those  who  had  expertise  in  the  task 
context  (or  who  thought  they  did)  tended  to  use  the  heuristic  less  and 
to  perform  worse  than  those  without  expertise  (ARKE0O). 

2.6  Summary  of  Relevant  Literature 

Group  deoision  processes  vary  depending  on  the  task,  The  group 
decision  process  that  best  fits  aotual  group  performance  for  dynamic 
intellective  tasks  of  the  sort  used  in  this  research  is  ‘truth, 
support'd,  wins'  (MCGR82),  Uncertainty  in  the  task  affects  group 
decision  performance  in  that  people  are  biased  in  their  perceptions  of 
uncertainty,  and  they  tend  to  adopt  less  than  optimal  strategies  for 
risky  choice  problems  (L1TT8C3.  Debiasing  methods,  such  as  providing 
task  training  using  decision  rules  (heuristics),  lead  deoision  groups  to 
adopt  more  appropriate  decision  strategies,  and  thus  positively  lmpaot 
decision  performance  CFISC823 . 

A  considerable  body  of  researoh  has  focused  on  supporting  decision 
making  through  the  presentation  of  Information  via  computer  graphics, 
Research  results,  especially  from  early  studies,  have  been  mixed,  but  it 
Is  clear  that  task  differences  have  an  impact  on  how  effectively 
graphics  can  be  used  to  support  deoision  making  CDI0K8Q],  A  graphic 
representation  has  been  established  as  superior  to  a  tabular 
representation  for  presenting  information  in  the  experimental  task 
chosen  for  this  research  [WILSB7),  Feature  integration  theory  suggests 
that  varying  the  nature  of  the  graphic  representation  used  in  the  task 
should  impact  the  information  processing  demands  on  the  decision  makers 
and  thus  Impact  group  performance  CTBB1 80 ,  TRE102], 

The  literature  cited  in  this  ohapter  provided  a  general  foundation 
for  this  study,  More  specific  ties  betwssn  ths  literature  and  the 
hypotheses  for  this  research  will  be  identified  in  the  following  chapter. 
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3.0  CONCEPTUAL  MODEL,  VARIABLE  SELECTION,  AND  RESEARCH  HYPOTHESES 

Tha  reviewed  literature  data  tha  foundation  for  developing  a 
conceptual  modal  of  tha  raaaaroh  laauaa  to  ba  addraaaad,  for  aeleoting 
apadflo  varlablas  to  atudy,  and  for  identifying  raaaaroh  queationa 
about  how  tha  variablaa  ara  ralatad,  Tha  flrat  aaotlon  of  thla  ohaptar 
explalna  tha  oonoaptual  modal;  it  la  followad  by  a  aaotlon  providing  tha 
rationala  for  ohooaing  variablaa.  (A  diaouaaion  of  how  tha  variablaa 
ara  oparationaliaad  followa  in  Ohaptar  4.)  Tha  third  aaotlon  of  thla 
chaptar  identified  tha  raaaaroh  hypothaaaa, 

3.1  Conceptual  Modal 

A  viaual  repredentation  of  tha  oonoaptual  modal  id  prodented  in 
Figure  3.1  Tha  daolalon  environment  included  tha  nature  of  tha  deoielon 
teak  aa  wall  aa  tha  phyaioal  and  aooial  featured  of  tha  context  in  which 
the  deolaion  la  made.  The  drouo  deoielon  prooadd,  at  tha  heart  of  tha 
modal,  entalld  the  aot  of  prooadurad  by  which  tha  group  operated  on  tha 
available  information  to  arrive  at  a  deolaion.  Varlatlona  in  tha 
content,  time  of  praaantation,  predentation  form,  and  other  featured  of 
the  information  available  to  tha  group  may  impact  tha  deolaion  proeuaa, 
Tha  deolaion  prooadd  adopted  may  aldo  ba  influenced  by  tha  predenoe  or 
abaanoa  of  a  deolaion  atratalv  iuiiaatad  by  outaida  aouroaa  (management, 
regulatory  aganoiad,  etc,),  Deolaion  outoomed  Include  tha  ohoload  made 
along  with  tha  feellnga  and  peroeptlona  of  thoee  involved  in  making  tha 
deolaion.  Tha  modal  elementa  ara  addraaaad  further  in  tha  following 
aubaaotiona. 


FIGURE  3,1.  Oonoaptual  Modal 


3.1.1  Decision  Environment  t, 

Ai  indicated  in  Chapter  3,  talk  differences  hava  had  a  major 
influence  on  raiaaroh  raiuiti.  Tha  experimental  talk  uaad  in  this  study 
was  a  dynamic  intailactiva  task  involving  uncertainty  and  various  lavals 
of  tima  prassura.  All  altarnativas  presented  to  tha  daoision  making 
group  raprasantad  potential  gains  (no  risk  of  loss) ,  soma  of  which  wars 
certain  and  soma  probabilistic)  A  detailed  description  of  the  task  is 
Included  In  Chapter  4. 

The  physical  features  of  the  decision  environment  (lighting) 
seating  arrangement)  equipment  layout)  ate.)  ware  controlled.  Tha 
social  context  was  controlled  only  to  the  extent  that  each  group 
received  the  same  task  orientation,  had  the  lame  opportunity  to 
communicate)  and  interacted  with  the  same  researcher.  Mo  attempt  was 
made  to  constrain  the  composition  of  the  teams  in  terms  of  personality, 
aptitude,  etc, 

3.1.2  grm  .Bwlilgn.  ,f.mm 

Huber  has  reviewed  several  models  of  group  decision  behavior  which 
could  explain  the  process  of  group  decision  making  CHUBE8 1 ] .  The 
rational  model  is  baled  on  an  allumption  that  group  members  use  the 
available  Information  rationally  and  make  their  decisions  on  the  basis 
of  the  expaoted  utility  of  the  alternatives  to  the  group  as  a  whole. 
The  expectation  under  a  second  model,  the  politioal/competltive  model, 
is  that  the  group  members  act  as  Individuals  who  enter  the  decision 
making  process  with  the  intent  nf  influencing  the  group  for  their  own 
benefit  rather  than  that  of  the  entire  group.  A  third  model,  the 
garbage  can  model,  emphasises  ohanoe  and  timing  in  the  decision  making 
process.  It  states  that  decisions  oeour  at  the  intersection!  of 
problems  looking  for  solutions,  solutions  looking  for  problems,  and 
decision  makers  looking  for  opportunities  to  make  decisions.  Finally, 
the  program  model  holds  that  decisions  are  the  natural  consequences  of 
existing  group  standards  and  norms  and  the  background,  training,  and 
biases  of  the  group  members. 

All  but  the  program  model  were  discarded  for  use  in  the  conceptual 
modal  to  describe  the  group  decision  process.  First,  as  indicated  in 
Chapter  3,  decision  groups  tend  not  to  behave  rationally,  especially 


under  conditions  of  uncertainty.  Next,  since  performance  was  scored 
only  on  a  team  basis,  there  was  nothing  obvious  to  be  gained  by  any 
individual  from  a  competitlve/politioal  approaoh  to  the  task.  Finally, 
over  the  course  of  this  study,  each  team  experienced  comparable  effects 
of  chance  and  timing  on  their  decision  alternatives,  so  any  differences 
in  decision  outcomes  could  not  be  explained  by  the  garbage  can  model. 
The  suggested  decision  strategy  used  in  this  study  was  designed  to 
overcome  group  member  biases  (e.g.,  bias  toward  certain  gains  over 
uncertain  gains  of  equal  or  higher  expeoted  utility)  in  porforming  a 
particular  dynamic  intellective  task.  This  researoh  attempts  to  show 
that  the  decision  processes  of  teams  without  the  suggested  decision 
strategy  may  be  described  using  Huber's  program  model  and  that  use  of 
the  suggested  strategy  changes  decision  processes  by  overcoming  biases, 
3,1.3  Information 

Information  content,  availability,  and  presentation  media  were  held 
oonstant  for  all  groups  in  this  study,  but  information  presentation  form 
varied,  A  presentation  form  that  places  lower  demands  on  the  group's 
information  processing  capacity  than  another  form  should  reduce  the  need 
for  task  training,  Aiding  the  identif loatlon  and  location  of  high  value 
alternatives  through  appropriate  presentation  form  may  also  help  to 
overcome  some  biases,  Information  presentation  form,  then,  may 
influence  the  group  decision  process  and  ultimately  have  an  impact  on 
decision  outcomes. 

Suggested  Decision  Strategy 

.  Hackman  and  Morris  propose  that  the  group  strategy  for  carrying  out 
a  task  (the  collective  choices  made  about  how  the  group  will  proceed) 
controls  a  major  portion  of  the  variation  in  group  decision  performance. 
A  team  may  either  implicitly  or  explicitly  develop  its  own  strategy, 
which  could  be  either  a  good  or  a  poor  strategy,  or  it  may  adopt  a 
strategy  developed  outside  the  group  (Imposed  by  management,  recommended 
by  consultants,  etc,)  [HACK83J.  A  suggested  deoision  strategy  that  is 
adopted  by  a  group  constitutes  a  direct  intervention  in  the  group 
deoision  process.  If  Huber's  program  model  holds,  the  intervention 
could  affect  the  decision  process  by  overcoming  biases  of  group  members 
and  setting  new  group  norms. 


3.1.5  Decision  Outcomes 


1 

The  most  obvious  outcome  of  the  group  decision  process  is  the 
decision  itself.  Ohervany,  Dickson,  end  Koaar  t D10K77 3  suggest  thet  the 
effectiveness  of  the  decision  process  mey  be  measured  In  terms  of 
decision  quality,  cost,  profit,  time,  eto.  For  the  dynamic  intellective 
task  used  in  this  research,  the  primary  measure  of  decision 
effectiveness  was  a  composite  of  the  correctness  and  timeliness  of 
decisions,  Other  decision  outcomes  include  developed  or  revised 
decision  strategies,  evolved  patterns  of  communication  and  influence 
among  the  group  members,  Individual  perceptions  of  the  task  and  of  other 
group  members,  and  so  on.  For  this  investigation,  particular  attention 
was  paid  to  perceptions  of  the  workload  Imposed  by  the  task, 

The  elements  of  the  conceptual  model  and  the  relevant  literature 
suggest  a  set  of  possible  research  variables,  The  variables  chosen  for 
this  study  are  identified  in  the  next  section, 

3.2  Expej  1  n.tftl  ..XltiAfrA.fi. 

Figure  3.3  categorises  the  experimental  variables  as  independent, 
dependent,  or  controlled.  The  variables  in  each  category  are  addressed 
in  the  following  subsections, 

3.2.1  Midi  nlinL  viiIfAL*  t 

Ohervany,  Dickson,  and  Xosar  Identified  a  set  of  independent  • 
variables  that  have  an  Impact  on  decision  quality.  The  Independent 
variables  include  the  decision  maker,  the  deoision  environment,  and  the 
characteristics  of  the  information  system  used  to  support  the  decision 
process  [DXCK77,  IVESOO].  The  group  deoision  process  itself  has  also 

been  Identified  as  an  important  variable  often  ignored  in  M18 

experimental  research  [BENBB4L  As  Indicated  by  the  research  purpose 
statement  provided  in  Chapter  2,  this  study  is  focused  on  facilitating 
decision  performance  through  graphic  information  presentation  and 
decision  heuristics.  In  other  words,  the  primary  independent  variables 
involve  the  information  presentation  form  (a  characteristic  of  the 
Information  system)  and  a  suggested  decision  strategy  (a  direct 
intervention  in  the  group  decision  process),  Two  additional  variables 
suggested  by  the  literature  were  time  stress  and  practice, 
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FIGURE  3.3.  Experimental  Variable* 


3.2. 1.1  Information  Preientatlon  Form.  Two  distinct  eontraete  relating 
to  information  presentation  form  are  addressed  in  this  study.  First, 
based  on  feature  integration  theory,  single-featured  codes  are 
contrasted  with  codes  that  use  conjunctions  of  features.  Feature 
integration  theory  indicates  that  performance  on  a  visually  oriented 
task  involving  location  and  identification  of  objects  varies  with  the 
form  in  which  the  objects  arc  presented  CTBEI60 ] .  Specifically, 
performance  using  conjunctively  coded  objeots  differs  from  performance 
using  single-featured  objects.  The  second  contrast  addressed  by  this 
study  is  between  two  single-featured  visual  codes.  Christ  and  others 
have  shown  that  object  identification  performance  varies  with 
differences  among  single-featured  coding  schemes  [CHRI75,  CHR183, 
IVES82] .  To  examine  both  contrasts  in  this  study,  information 
presentation  form  was  varied  between  groups,  each  decision  making  team 
receiving  information  presented  via  a  color  (single-featured),  an 
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alphabetic  (single-featured) ,  or  a  combined  color  and  alphabetic 
(conjunctive)  coding  scheme. 

3. 2. 1.2  Suggested  Decision  Strategy.  For  this  Study,  half  of  the 
groups  were  left  to  develop  their  own  strategy  and  half  were  provided 
with  and  encouraged  to  use  an  experimenter-developed  strategy.  The 
suggested  strategy  took  the  form  of  a  set  of  four  deoision  heuristics 
(see  Chapter  4)  which  should  lead  to  consistently  good,  although  not 
necessarily  optimal,  performance  on  the  task. 

3. 2. 1.3  Time  Stress,  Previous  studies  using  very  similar  experimental 
tasks  have  shown  that  performance  declines  under  increased  levels  of 
time  stress  (BR0W8B,  W1LSB7],  Time  stress  Should  have  the  same  effect 
in  this  study.  The  real  Intent  in  including  time  stress  as  a 
variable,  however,  is  to  examine  its  interaction  with  the  primary 
independent  variables,  Information  presentation  form  and  suggested 
decision  strategy,  Are  performance  differences  associated  with 
variations  in  the  primary  variables  any  more  or  less  evident  under 
various  task  conditions?  One  mechanism  for  increasing  attention  loads 
(task  difficulty)  in  a  visual  identification  task  is  to  inorease  the 
speed  at  which  objeots  are  presented  (increase  time  stress),  This  study 
compared  performance  within  groups  at  two  levels  of  time  stress.  The 
rate  at  which  objeots  were  presented  in  half  of  the  experimental  trials 
was  twice  as  high  as  in  the  other  half;  the  sequence  of  high  stress  and 
moderate  stress  trials  was  randomised. 

3. 2. 1.4  Practice.  According  to  the  Power  Law  of  Practice,  mental  task 
performance  improves  with  practice.  More  specifically,  the  greatest 
improvements  are  achieved  in  the  early  stages  of  learning  with 
performance  leveling  out  over  time  (0ARD83).  Performance  on  the  TRAP, 
then,  should  improve  with  practice,  especially  in  early  sessions,  as  it 
has  in  previous  related  studies  CBRQW88,  WILS871,  Because  of  the 
potential  effects,  there  have  been  calls  to  include  practice  or  learning 
as  an  independent  variable  in  graphios  research  (JARVBSbl.  As  was  the 
case  for  the  time  stress  variable,  however,  the  praotioe  variable  is 
included  primarily  to  examine  its  interactions  with  the  presentation 
form  and  deoision  strategy  variables.  In  this  study,  each  team 
completed  a  training  session  and  a  total  of  32  experimental  trials 
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arranged  in  4  sessions  of  9  triala  each.  The  design  provided  ^for  a 
wlthin-group  assessment  of  practice  effects  over  the  four  experimental 
sessions . 

3.2.2  Dependent  Variables 

The  decision  outcomes  of  interest  in  this  study  were  deoision 
quality  and  perceptions  of  workload.  For  the  dynamic  intellective  task 
used  in  this  research,  both  deoision  correctness  and  decision  making 
time  were  reflected  in  measures  of  decision  quality.  Perceptions  of  the 
workload  imposed  by  the  task  were  captured  in  terms  of  perceived  time 
stress,  mental  effort,  and  psychological  stress. 

3.22.1  Decision  Quality.  High  quality  decisions  in  performing  the 
experimental  task  depend  on  both  speed  and  accuracy  in  identifying  and 
committing  resources  to  high  value  events.  Therefore,  performance  was 
measured  in  terms  of  efficiency  (scores  amassed  in  given  amounts  of 
time),  Specifically,  two  types  of  deoision  quality  scores  were  examined 
and  compared  across  treatments:  ratios  of  the  average  points  earned  by 
the  teams  to  the  points  earned  by  a  ‘model*  decision  maker  and 
proportions  of  various  types  of  events  completed. 

3. 2. 2. 2  Perceived  Workload.  Data  on  perceived  workload  was  collected 
using  the  Subjective  Workload  Assessment  Technique  (SWAT)  (see  Appendix 
A  for  a  description  of  SWAT).  Immediately  following  each  experimental 
trial,  each  participant  rated  the  trial  in  terms  of  its  time  stress,  its 
demands  on  mental  effort,  and  its  psychological  stress  (level  of 
confusion,  dlstraation,  etc.).  The  three  ratings  were  weighted  and 
combined  to  form  an  overall  measure  of  perceived  workload.  Individual 
SWAT  scores  were  then  averaged  to  generate  a  group  measure  of  perceived 
workload ;  team  data  was  compared  across  treatments. 

3.2.3  Controlled  Variables 

Some  variables  that  were  not  chosen  as  independent  variables  for 
this  study  are  known  to  affect  group  deoision  performance.  Those 
variables  (group  sise,  the  deoision  objective,  the  physical  features  of 
the  task  environment,  and  the  information  provided  to  the  decision 
makers)  were  controlled  to  minimise  their  influence  on  the  group 
decision  process. 
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3.2.3. 1  Group  Size.  Group  size  wad  held  constant  at  three  individuals 
per  team. 

3. 2. 3. 2  Deoision  Objective.  The  deoision  objective  (the  purpose  or 
goal  toward  which  deoision  makers  direot  their  deoision  behavior)  is  a 
variable  that  can  influence  the  deoision  process  (CLAR031.  The  deoision 
objective  in  the  experimental  task  was  to  maximise  total  points  earned 
by  the  team.  A  statement  of  the  objeotive  was  lnoluded  in  the  training 
provided  to  all  participants  (see  Chapter  4). 

3. 2. 3. 3  Physical  Environment.  The  lighting,  seating  arrangement,  and 
physical  layout  of  the  experimental  facility  was  identical  for  all 
teams. 

3. 2. 3. 4  Information  Content.,  Availability,  and  Presentation  Media. 
Huber  lists  the  following  data  requirements  for  rational  decision  making 
[HUBEBll: 

--  What  are  the  alternatives? 

--  What  are  the  future  conditions  that  might  be  encountered? 

--  What  are  the  probabilities  of  the  future  conditions? 

--  What  are  the  criteria  to  be  used  in  evaluating  alternatives? 

What  are  the  relative  importances  of  the  various  criteria? 

--  What  are  the  payoffs,  or  oosts,  associated  with  various 
outcomes? 

--  What  are  the  constraints  on  the  payoffs  or  costs? 

Presentation  forms  varied,  but  the  oontent  and  timing  of  information  to 
meet  each  of  the  requirements  was  the  same  for  all  groups  in  this  study. 
The  experimental  task  was  presented  to  all  teams  via  individual  CRT's. 
Event  sequences  were  randomised,  but  the  number  of  events  per  trial  and 
the  mix  of  event  typos  was  constant  for  every  trial  and  was  identical 
lor  all  teams. 


3.3  Research  Hypotheses 

The  preceding  section  on  experimental  variables  identified 
information  presentation  form  and  deoision  strategy  as  the  primary 
independent  variables  in  this  research.  The  plan  for  the  research  was 
to  measure  the  effect  of  varying  those  factors  on  two  deoision  outcomes! 
decision  quality  and  perceived  workload,  The  following  subsections 
provide  sets  of  hypotheses  that  anticipate  the  effects  of  eaoh  of  the 


independent  variable*  and  thnir  key  interaction*  on  deci*ion  quality  and 
perceived  workload.  The  hypotheiei  in  aeotion  3.3.9  addreei  a  comparison 
ol  the  effect*  of  information  presentation  form  and  suggested  deoiaion 
strategy  on  deoiaion  quality  and  peraaived  workload, 

3.3.1  Effecta  of  Information  Presentation  Form 

Experimental  oompariaon*  among  a  variety  of  Information 
praaantation  form*  are  valuable  in  that  the  reault*  lead  to  praotioal 
guideline*  for  *y*tem  development.  Thi*  raaearch  compare*  the  impaot  of 
three  coding  «oheme*i  two  single-featured  aohemea  (color  and 
alphabetic)  and  a  conjunctive  icheme  (combination  of  color  and 
alphabetic) .  A  clear  advantage  for  one  echeme  would  lead  to 
racommendationa  for  that  acheme  in  designing  ayatem*  for  team  deciiion 
making  in  almilar  deoiaion  environment*.  If  no  clear  difference  among 
the  acheme*  emerge*,  ayatem  designer*  have  a  more  open  aholae,  The 
following  two  aubaection*  present  hypotheses  concerning  the  effect*  of 
information  presentation  form  on  decision  quality  and  perceived 
workload. 

3.3. 1.1  l o,miUg.n..£iiiftnA*.U 9n.  in  test*  of 
a  feature-integration  theory  of  attention,  performance  on  visual  talk* 
involving  location  and  identification  ol  object*  was  altered  by  changing 
the  feature*  used  to  distinguish  object*  in  the  visual  display  (TRE1B01. 
The  theory  suggest*  that  people  perform  visual  *earoh  task*  more  quickly 
and  accurately  when  the  cue*  vary  on  a  single  dimension  rather  than  a* 
conjunction*  of  feature*.  The  symbol*  used  to  represent  event*  in  a 
baseline  Study  using  the  Team  Resource  Allocation  Problem  (TRAP) 
[BR0W95]  appeared  as  conjunctions  of  separable  features  (shape  and 
color).  If,  instead,  symbols  were  to  vary  within  a  single  feature 
(e.g.,  color)  teams  should  be  able  to  recognise  events  without  having  to 
focus  attention  on  each  symbol  serially,  Therefore,  they  should 
identify  and  choose  the  most  beneficial  events  more  readily  and  achieve 
higher  scores  than  those  team*  who  must  attend  to  oonjunotively  coded 
symbols. 

In  a  review  of  color  coding  research,  Christ  (CHRI73)  examined  the 
results  of  a  number  of  studies  that  reported  data  on  the  accuracy  and 
speed  of  identifying  single-featured  stimuli,  He  concluded  that  color 
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is  superior  to  size,  brightness,  end  shape  but  inferior  to  letters  for 

accuracy  in  identification  tasks.  On  the  other  hand,  color  provided  a 

speed  advantage  over  letters  for  locating  objeots  in  a  display.  Qhrist 

later  confirmed  that  location  times  were  shorter  for  colored  dots  than 

for  letters  in  dense  displays  (13-objeot  displays  as  contrasted  to  4- 

object  displays)  C0HR183 ] .  Since  performance  in  the  TRAP  depends  on 

both  accuracy  and  anted  in  identifying  the  event  symbols,  the  results 

reported  by  Christ  provide  no  olear  expectation  of  an  advantage  in  TRAP 

performance  for  either  alphabetic  or  oolor  coding. 

The  combined  expectations  for  the  effects  of  information 

presentation  form  on  decision  quality  based  on  feature  Integration 

theory  and  the  results  of  previous  researoh  with  single-featured  visual 

codes  may  be  stated  in  hypothesis  form: 

Hla.  The  quality  of  team  decisions  in  the  TRAP  will  be  higher 
for  teams  confronted  with  Single-featured  (either  alphabetic 
or  color-coded)  event  symbols  than  for  teams  confronted  with 
events  represented  by  conjunctions  of  features.  However, 
there  will  be  no  difference  in  decision  quality  between  teams 
using  an  alphabetic  coding  Scheme  and  teams  using  a  oolor 
coding  scheme. 

3.3. 1.3  Information  Presentation  Form  and  Perceived  Workload.  Beoause 
'operators  tend  to  adapt  to  tasks  and  hold  performance  constant  over  a 
broad  range  of  cond4tionS,'  CLEMA0O,  p,  038]  performance  measures  may 
not  refleot  the  full  impact  of  the  information  presentation  form 
treatments.  Although  performance  differences  may  be  minimal  or 
nonexistent,  however,  the  subjects  may  still  perceive  differences  in 
mental  workload.  Subjective  measures  of  mental  workload  have  been  Shown 
to  be  sensitive  to  objective  changes  in  task  workload  ILEMA80], 
Therefore,  they  represent  alternative  and  potentially  more  Sensitive 
measures  of  the  impact  of  differences  in  information  presentation 
treatments  in  the  TRAP. 

Since  subjective  workload  measures  are  alternate  measures  of  the 
same  effects  rather  than  measures  of  different  effects,  it  is  assumed 
that  factors  that  lead  to  better  performance  should  also  lead  to 
perceptions  of  lower  workload.  Specifically,  since  feature  integration 
theory  states  that  people  perform  visual  tasks  more  quickly  and 
accurately  when  the  cues  vary  on  a  single  dimension  rather  than  as 
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conjunction*  of  feature*,  it  i*  reasonable  to  expect  that  people ^using 
single-f eatured  cue*  would  also  perceive  lower  level*  of  mental 
workload,  Since  color  and  alphabetic  coding  scheme*  seem  to  trade  off 
speed  for  accuracy  in  visual  searoh  tasks,  no  differences  in  perceived 
workload  are  expected  between  the  two  single-featured  coding  aohemes. 
Thus,  the  following  hypothesis  concerning  perceived  workload  parallels 
that  concerning  team  deoision  quality) 

Hlb,  The  perceived  workload  in  the  TRAP  will  be  lower  for 
teams  confronted  with  single-featured  (either  alphabetic  or 
color-coded)  event  symbols  than  for  teams  confronted  with 
events  represented  by  conjunctions  of  features.  However, 
there  will  be  no  difference  in  perceived  workload  between 
teams  using  an  alphabetic  ooding  scheme  and  teams  using  a 
oolor  coding  soheme, 

3.3.2  Effects  of  Sugie* 


Debiasing  methods  (techniques  for  reduoing  human  bias  in  decision 
making)  include  providing  training  for  the  decision  task  [F1S062). 
Providing  decision-aid  heuristie*  as  a  form  of  task  training  has  been 
shown  to  Improve  decision  performance  on  an  ill-structured  career 
planning  task  (0ATS87).  As  an  intellective  task,  the  TRAP  is  a  task 
with  demonstrably  correct  choices  IMOQR04].  Prescriptive  heuristics  for 
the  TRAP  were  developed  to  overcome  specific  biases  evident  in  the 
performance  of  previous  participants  in  the  TRAP  (See  Chapter  4);  1 , e . , 
the  heuristics  lead  the  decision  makers  to  reoognla*  the  demonstrably 
correct  choices.  The  following  two  subsections  present  hypotheses 
concerning  the  effects  of  a  suggested  deoision  strategy  in  the  form  of 
deaision-aid  heuristics  on  decision  quality  and  perceived  workload. 
3'3-a.l  Suggested  Decision  Strategy  and  Decision  Quality.  It  has  been 
shown  that,  for  intellective  tasks,  the  best-fitting  model  of  a  deoision 
rule  is  'truth  supported  wins*  (if  at  least  two  group  members  recognise 
the  right  answer,  the  group  will  accept  it  as  correct)  t MCQR84 ] . 
Providing  deoision  heuristics  should  increase  the  probability  that  at 
least  two  group  members  will  recognise  the  oorreot  choices  in  the  TRAP 
and  should  lead  to  improved  deoision  performance, 

,  An  indirect  benefit  of  providing  deoision  heuristics  follows  from 
the  explicit  attention  paid  to  a  decision  strategy.  Hackman  and  Morris 
propose  that  the  strategy  (tho  collective  ohoicel  made  about  how  the 
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group  will  proceed)  used  by  group  members  in  carrying  out  the  task  is  a 

variable  that  controls  a  major  portion  of  the  variation  in  group 

decision  performance.  There  seems  to  be  a  pervasive  norm,  however, 

against  explicitly  discussing  the  group  strategy  in  spite  of  the  faot 

that  group  performance  seems  to  Improve  with  explicit  attention  to  the 

strategy  CHACK83 ] .  Providing  a  set  of  decision  heuristios  to  a  deoision 

making  group  and  encouraging  a  discussion  about  the  heuristics  should, 

therefore,  lead  to  improved  decision  performance. 

H2a(l),  The  quality  of  team  decisions  in  tho  TRAP  will  be 
higher  with  provision  and  discussion  of  deoision  heuristios 
than  without  provision  and  discussion  of  decision  heuristios. 

A  particular  aspect  of  decision  quality  of  special  interest  to  this 

study  is  the  quality  of  decisions  made  under  uncertainty,  (Note  that 

this  aspect  of  deoision  quality  was  not  addressed  in  the  discussion 

about  the  effects  of  information  presentation  form.  Binoe  uncertain 

events  in  the  experimental  task  are  presented  to  all  decision  teams  in 

identical  formats  (see  chapter  4),  there  should  be  no  presentation 

impact  on  decisions  about  uncertain  events.) 

.Expected  utility  theory  states  that  rational  decision  makers,  given 
tho  possible  payoffs  of  the  alternatives  and  the  probabilities  of 
various  states  of  nature,  choose  among  alternatives  on  the  basis  of 
expected  utility  (the  produot  of  the  probability  of  an  outcome  and  its 
potential  utility)  [KAHN791,  In  a  number  of  studies,  however,  deoision 
makers  have  tended  to  favor  oertain  outcomes  over  merely  probable 
outcomes  with  equivalent  or  even  higher  expected  value  (an  aspect  of 
prospect  theory)  (KAHN79).  Prospect  theory,  then,  would  suggest  that 
teams  would  undervalue  uncertain  events  in  the  TRAP.  The  'choice  shift' 
phenomenon  in  groups,  the  tendency  for  groups  to  gamble  more  or  less 
than  their  individual  members  would  in  making  the  same  deoision  [FISH80, 
MUHS81 ] ,  suggests  that  teams  could  either  undervalue  or  overvalue 
uncertain  events  in  tho  TRAP,  The  direction  of  difference,  therefore,  is 
difficult  to  predict,  In  tho  absence  of  provision  and  discussion  of 
decision  heuristios,  however,  teams  should  seek  additional  information 
about  and  then  commit  resources  to  uncertain  events  in  the  TRAP  at  a 
rate  different  from  that  anticipated  under  expected  utility  theory. 
Since  the  heuristics  provided  as  a  suggested  decision  strategy  are  based 
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in  pari  on  expected  utility  theory,  however,  teems  receiving  and 

discussing  the  heuristics  ahould  choose  to  oonmit  raaouroaa  to  uncertain 

events  according  to  axpaotad  utility  thaory. 

H2a (2) .  Without  tha  proviaion  and  diiouaaion  of  dacialon 
*  hauriatica,  taama  will  aaak  additional  information  about  and 
commit  raaouroaa  to  unoartain  avanta  at  a  rata  diffarant  from 
that  antiolpatad  according  to  axpaotad  utility  thaory.  With 
tha  proviaion  and  diaouaalon  of  daolaion  hauriatioa,  taama 
will  aaak  additional  information  about  and  oonunit  raaouroaa  to 
unoartain  avanta  according  to  axpaotad  utility  thaory. 

3. 3.2. 2  Suggested  Dacialon  Strategy  and  Paroaivad  Workload.  Since  tha 

auggaatad  dacialon  atratagy  ia  axpaotad  to  inoreaee  tha  probability  that 

at  laaat  two  group  members  will  recognise  tha  oorraot  choloaa  and  since 

tha  group  will  know  that  they  ahara  a  oommon  atratagy.  taama  raoaivlng 

tha  hauriatica  ahould  perceive  a  lower  level  of  workload  in  tha  TRAP. 

H2b.  Tha  perceived  workload  in  tha  TRAP  will  be  lower  with 
proviaion  and  diaouaalon  of  daolaion  hauriatica  than  without 
proviaion  and  diaouaalon  of  daolaion  hauriatioa. 

3.3.3  Effaota  of  Tima  Straaa 

Among  tha  dimensions  on  which  a  daolaion  task  oan  vary  are  task 
content,  task  difficulty,  and  task  structure  tDIOKBOh  Por  this 

research,  oommon  task  content  and  task  structure  ware  maintained  lor  all 
participants  in  all  sessions,  In  order  to  detect  soma  of  tha  influence 
of  the  task  environment  on  performance,  task  difficulty  was  varied.  One 
mechanism  for  increasing  task  difficulty  in  a  visualidentif ioation  task 
is  to  lnaraasa  tha  spaed  at  which  objects  are  presented  (increase  time 
stress) . 

3.3.3. 1  Tima  Stress  and  Decision  Quality.  Previous  studies  using  vary 

similar  experimental  tasks  have  shown  that  parformanoa  declines  under 

increased  levels  of  time  stress  tBROWSB,  WILS87].  Tima  stress  ahould 

have  tha  same  affect  in  this  study. 

H3a.  Tha  quality  of  team  decisions  in  tha  TRAP  will  be  higher 
under  moderate  time  pressure  than  under  high  time  pressure, 

3. 3. 3. 2  Tima  Stress  and  Perceived  Workload.  Particularly  axnce  one  of 
the  three  components  of  the  workload  measure  used  focuses  on  time 
stress,  perceptions  of  workload  in  the  TRAP  Should  increase  under 
increased  time  stress. 
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H3b,  Thi  perceived  workload  in  the  TRAP  will  be  lower  und«{ 
moderate  time  pressure  than  undar  high  time  prtiaura. 

3.3.4  Effects  of  Practice 

Baoauia  of  the  pobantial  affaobl  of  learning  on  bank  parforroanoa  in 
baaka  uling  graphic  rapraaanbabiona ,  thara  hava  baan  call*  bo  axamina 
praobica  or  laarning  affaobl  in  graphic!  raiaaroh  tJARVBBbl. 
Participant!  in  bhil  study  complabad  a  tobal  of  33  axparlmanbal  trial! , 
providing  opportunity  bo  dataob  bha  impact  of  laarning  on  parformanca. 

3.3.4. 1  Practica  and  Daolilon  Quality.  According  to  tha  Powar  Law  of 
Praetioa,  manbal  talk  parformanca  improvaa  with  praobioa.  Mora 
ipacif ioally,  tha  graatait  lmprovamanti  ara  achlavad  in  tha  aarly  itagaa 
of  laarning  with  parformanoa  lavaling  out  ovar  tima  (0AR0B3]. 
Parformanca  on  tha  TRAP,  than,  ahould  lmprova  with  practice,  aipaoially 
in  aarly  sessions,  ai  it  hai  in  pravioua  ralatad  itudlaa  tBROWBB, 
W1LSB7  ] , 

H4a.  Tha  quality  of  taam  daciiiom  in  tha  TRAP  will  lmprova 
with  practica,  aipaoially  in  aarly  laiiloni. 

3. 3. 4. 3  Practica  and  Parcalvad  Workload.  With  practica,  daoliioni  in 

tha  TRAP  ihould  bacoma  ralativaly  reutlna.  Parcaptloni  of  workload, 

than,  ihould  dacraaia  at  a  daoraaiing  rata. 

H4b.  Tha  paroaivad  workload  in  tha  TRAP  will  daclina  with 
praobica,  aipaoially  in  aarly  laflioni. 

3.3.o  MittgilH-lX MKroiUMiEmi iiUUga 
Form 

Ona  primary  inbaralb  in  axamlning  bha  affaobl  of  tima  ibraii  in 
bhil  raiaaroh  oonoarnl  tha  interactive  affaobl  of  tima  atraii  and 
information  praianbatlon  form.  Tha  impact  of  information  praiantation 
lorm  could  vary  dapanding  on  talk  difficulty,  operationalised  in  bhil 
study  aa  tima  ibraii, 

3.3.5. 1  Tima  Btram/Inf ormatlon  Praianbatlon  Form  and  Daolilon  Quality. 
Sines  tima  praiaura  would  oonitraln  bha  tima  available  to  focui  on 
individual  lymboll,  taami  that  can  racogniaa  lymboli  in  parallel  ihould 
hava  an  incraaiing  advantage  ai  time  praiaura  increases  ovar  teams  that 
must  attend  bo  lymboll  serially.  Trailman  and  Qelade  alio  report  that 
operating  with  conjunctively  coded  objects  undar  time  constraint!  leads 
to  fall*  positive  errors  (illusory  conjunctions)  (TRE180).  Therefore, 


the  advantage  of  aingie-lea.ured  aymbola  over  conjunctively  jOoded 
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aymbola  ahould  be  nor*  >,ronounaea  whan  Urn*  preaaure  inor*»i*i. 

Christ  COHRI70]  *.o,iou‘ded  tVet  th*  relative  *1  feotiveneaa  ol 
aingle-featured  (color  and  alph  nu>..*.rlo)  aymbola  for  accuracy  in 
identification  taaka  varied  wivh  .aak  difficulty.  For  example,  th* 
accuracy  advantage  for  alphanumeric  aymbild  inoraaita  with  Ineraaaea  in 
th*  number  of  itimull  to  b*  identifier  and  with  d*cr«ai«i  in  expoaur* 
tin*.  Th*  ao*«d  advantage  for  locating  or’ora  over  locating  lettera, 
however,  ahould  alao  be  more  lntenl*  under  high  time  preiaur*.  Bine* 
there  are  of faetting  foroea  allocated  with  time  preiaura.  it  la 
reaionabla  to  expect  that  th*  relative  ef  ieeWveneaa  of  alphabetioally 
coded  aymbola  and  oolor  coded  aymboltf  in  t..e  TRAP  ahould  not  vary  with 


inoreaaed  tin*  preaaur*. 

HQ*.  Tim*  preaaur*  and  preaentation  treatment  will  interact 
•uoh  that  the  advantage  of  amgle-featured  ooding  over 
'conjunctive  ooding  for  deoiaion  quality  will  be  greater  under 
high  time  preaaur*  than  under  moder*\e  time  preaaure. 
However,  th*  relative  effiotivenedi  of  alphabetic  and  oolor 
ooding  for  deoiaion  quality  will  remain  th*  ham,,  whether  under 
high  or  moderate  time  preaaure. 

3. 3. s. a  iim  ,-Eir.fliixid. 
Workload.  The  atrain  of  attempting  to  aerially  attend  to  each  aymbol 
under  inoreaaed  time  preaaure  ahould  effect  the  workload  perception!  of 
teama  using  conjunctively  coded  aymbola.  Therefore,  the  advantage  of 
aingle- featured  aymbola  over  conjunctively  ooded  aymbola  ahould  be  more 
pronounced  when  time  preiaur*  inoreaiea.  Offaettlng  foroea  relating 
time  preaaur*  to  aingl*-f eatured  codea,  however,  lead  to  an  expectation 
that  any  difference!  in  workload  peroeptlona  between  teama  using 
alphabetically  coded  aymbola  and  team!  ualng  oolor  ooded  aymbola  will 
not  very  with  inoreaaed  time  preaaure. 

HQb.  Tim*  preaaure  and  preaentation  treatment  will  interact 
auoh  that  th*  edvantege  of  aingle-featured  ooding  ovar 
conjunotlv*  coding  for  perceived  workload  will  be  greeter 
under  high  time  preaaure  than  undar  moderate  time  preaaure. 
However,  the  relative  impaot  of  aiphebetlo  and  ooior  ooding  on 
perceived  workload  will  remain  the  lame  whether  under  high  or 
moderate  time  preaaure. 
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3.3.8  Interactive  Effects  of  Practice  and  Information  Presentation  Form 
‘A*  nii  the  out  (or  tlino  stress,  a  primary  interest  in  examining 
the  •Ilooti  o(  praotioa  in  this  roaoaroh  concerns  the  interaotive 
effeots  o(  practice  and  information  proaontatlon  form.  Praotiot  oould 
mediate  tha  impart  o (  information  praaantation  form  on  ptrformanoa  and 
parcaivad  workload, 

3.3,6, 1  fract lea/ Information  Proaontatlon  Rorm  and  Daolalon  Quality, 
Baaauaa  othar  raaaarohara  had  found  qualitative  ohan|aa  in  parformanoa 
on  vidual  taaka  with  axtandad  praotioa,  Traiaman  and  Qalada  examined  tha 
effects  of  praotioa  on  aaarohin|  for  oonjunotivaly  oodad  objects,  They 
found  no  indication  of  movement  from  aerial  to  parallel  proceeding  over 
13  blocks  of  praotioa  CTRIIlOl.  Therefore,  parformanoa  diffaranoea 
between  conjunctive  and  dingle-featured  coding  iohamai  should  remain 
oonatant  avan  with  axtandad  praotioa, 

On  tha  othar  hand,  Ohriat  had  demonstrated  that  extended  praotioa 
with  a  vlaual  taak  tanda  to  attenuate  any  diffaranoea  in  parformanoa 
baaed  on  uae  of  different  dingle-featured  coded  (letterd,  digltdi 
familiar  geomatrio  ahaped,  and  oolorad  dote)  [0HRI75,  CHRI631,  In  othar 
worda,  although  no  overall  difforanoa  in  parformanoa  had  bean 
hypothaalaad  between  tha  alphabetic  and  oolor  oodlng  dchamad,  any 
diffaranoea  that  might  axldt  during  early  experimental  daddiona  would  be 
attenuated  with  praotioa. 

H6a,  Praotioa  and  pradantation  treatment  will  lntaraot  duoh 
that  any  differenead  in  tha  relative  effeotlveneea  of 
alphabetic  and  oolor  oodlng  for  daolalon  quality  will 
attenuate  over  tha  four  experimental  daddiona,  However, 
differenced  in  the  relative  of faotlvenedd  of  eonjunotlve  and 
single-featured  oodlng  will  remain  oonatant  over  the  four 
experimental  daddiona, 

3*3.0, a  Praotioa /In  for mat  ion  Preaentatlon  Perm  and  Perceived  Workload. 
If  the  mechanism  by  which  people  mudt  identify  oonjunotivaly  oodad 
dymbola  (aerially  attending  to  eaoh  symbol)  does  not  vary  with  praatlce, 
it  id  likely  that  diffaranoea  in  workload  perceptions  between  those 
using  conjunctive  and  those  using  single-featured  schemes  will  not  vary 
with  praotioa,  On  the  other  hand,  praotioa  should  make  users  equally 
comfortable  with  any  single-featured  coding  saheme,  attenuating  any 
early  dlfferenoes. 
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H0b.  Practice  and  praaantation  treatment  will  interact  auoh 
that  any  dlffarencea  in  peroeived  workload  aaaooiatad  with 
alphabatlo  and  oolor  coding  will  attanuata  ovar  tha  four 
experimental  laaaioni.  Howavar,  dilfaranoai  in  paroaivad 
workload  aaaooiatad  with  conjunctive  and  eingie-teatured 
coding  will  remain  oonatant  ovar  tha  four  axparlmantal 
aeaaiona. 

3.3.7  ]p.tjrt9UYi-6((l?>t-S,(_Ti'R.,.S^lLL->J.d_SugfU^ad  pa^^Oj) 
Strategy 

Tima  atraaa,  aa  a  oomponant  of  taak  difficulty,  la  axpeotad  to 
internet  not  only  with  information  praaantation  form  but  with  tha 
au'jaabad  daoiaion  atratagy,  Tha  Impact  of  tha  auggaatad  daoiaion 
atratagy  oould  vary  depending  on  tha  laval  of  time  atraaa, 

3.3.7. 1  Tima  Btraaa/8udiaatfd  Dtolalon  Strateiy  and  QaaiaUn  ftmOliy. 

The  proviaion  and  diacuaaion  of  haurlatica  ahould  routinixa  aoma 

daoiaiona  or  aetabliah  an  officiant  preoaaa  for  making  daoiaiona, 

Tharafora,  haurlatica  ahould  ba  avan  mora  banafloial  under  high  time 

atraaa  than  under  moderate  time  atraaa. 

H7a,  Tina  praaaura  and  haurlatica  treatment  will  intaraot 

auoh  that  tha  difference  in  declalon  quality  related  to 
haurlatica  treatment  will  be  greater  under  high  time  praaaura 
than  under  moderate  time  praaaura. 

3. 3. 7. 2  hm  liisiii&l MIlLsaA* 

Aoauming  that  tha  haurlatica  provided  are  almpla  and  limited  in  acopa  ao 
aa  not  to  induaa  atraaa  thamaelvea,  routiniaing  daoiaiona  through 
proviaion  and  diaouaalon  of  haurlatica  ahould  reduce  workload 
pnrcaptlona  avan  mora  under  high  time  atraaa  than  under  moderate  time 
atraaa. 

H7b.  Tima  praaaura  and  haurlatica  treatment  will  interact 

auoh  that  tha  difference  in  perceived  workload  related  to 
'haurlatica  treatment  will  ba  greater  under  high  time  praaaura 
than  under  moderate  time  praaaura. 

3.3.8  Interactive  Bffecta  of  Practice  and  Suagected  Daemon  Strategy 
Along  with  examining  tha  mediating  affecta  of  practice  on  tha 

impact  of  information  praaantation  form,  thle  itudy  addreaaaa  the 
Interactive  affecta  of  practice  and  tha  auggaatad  daoiaion  atratagy. 

Praotlca/Suileatad  Dtclalofl..  Strategy  and  Pedalon  Quality. 
Extended  practice  with  tha  TRAP  ahould  allow  taama  to  dlecover  and  adopt 
on  their  own  atrataglaa  that  approximate  the  benefita  of  the  auggaatad 
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;  decision  strategy.  If  so,  performance  differences  between  teams 

operating  with  and  those  operating  without  the  suggested  decision 

strategy  would  attenuate  with  praotioe, 

K8a.  Praotioe  and  the  suggested  decision  strategy  will 

interact  suoh  that  the  advantage  with  provision  and  discussion 
of  decision  heuristics  will  attenuate  over  the  four 
experimental  sessions. 

3 . 3 . 8 . 2  Praotlco/BuSSssted  Decision  Strategy  and  Perceived  Workload. 

As  teams  without  the  suggested  decision  strategy  discover  and  adopt 

strategies  that  routlniae  decision-making,  their  perceptions  of  workload 

should  approaoh  those  for  teams  with  the  suggested  decision  strategy. 

H6b.  Practice  and  the  suggested  deoislon  strategy  will 

interact  suoh  that  differences  in  peroeived  workload  between 
teams  with  and  those  without  provision  and  disousslon  of 
deolsion  heuristics  will  attenuate  over  the  four  experimental 
sessions. 

< 

3.3.0  Relative  Effects  of  Information  Presentation  Form  and  Suggfl.ttd 
Decision  Stratesy 

While  there  have  been  studies  about  the  effects  of  various 
presentation  forms  and  others  about  the  efteots  of  suggested  heuristics, 
there  is  very  little  research  evidenoe  concerning  the  relative  impact  of 
presentation  form  and  heuristics.  Oats-Baril  and  Huber  examined  the 
effects  of  decision-aid  heurlstios  on  decision  quality  in  an  ill- 
structured  aareer  planning  task.  In  their  study,  use  of  a  heuristic  had 
a  positive  impact  on  decision  quality,  but  oomputer  delivery  of  decision 
aids,  as  opposed  to  pencil-and-paper  delivery,  had  no  affect  ICATS07 ] . 
Given  the  limited  theoretical  and  research  basis  for  prediction,  the 
hypotheses  addressing  relative  impact  are  stated  here  in  the  null  form. 

3 . 3 . 8 . 1  Information  Presentation  Form  vs  Suggested  Decision  Strategy 
and  Decision  Quality. 

H9a,  The  magnitude  of  impaot  on  deoision  quality  related  to 
information  presentation  form  and  to  the  suggested  decision 
strategy  will  be  the  same, 

3. 3. 0.2  Information  Presentation  Form  vs  Suggested  Decision  Strategy, 
and  Perceived  Workload. 

HOb,  The  magnitude  of  impaot  on  peroeived  workload  related  to 
Information  presentation  form  and  to  the  suggested  deoision 
strategy  will  be  the  same, 
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3 . 4  Summary  of  Hypotheses  1, 

The  hypotheses  for  this  research  predict  that  information 
presentation  form,  suggested  decision  strategy,  time  pressure,  and 
practice  each  have  an  impact  on  both  deoislon  quality  and  perceived 
workload  in  a  dynamic  group  decision  task  involving  uncertainty,  The 
two  primary  independent  variables,  information  presentation  form  and 
suggested  decision  Strategy,  are  hypothesised  to  have  equal  impact  on 
decision  quality  and  perceived  workload.  Time  stress  and  praotloe  are 
hypothesized  to  have  a  modifying  influence  on  the  effects  of  information 
presentation  form  and  suggested  decision  strategy.  The  next  chapter 
explains  the  context  in  whioh  the  hypotheses  were  tested  by  describing 
the  experimental  task  and  the  procedures  iollowed  in  conducting  the 
research. 


4.0  RESEARCH  ENVIRONMENT  AND  METHOD  v 

Because  of  the  importance  of  task  variables  In  experimental 
research,  the  first  section  of  this  chapter  is  devoted  to  a  description 
of  the  experimental  task  used  in  this  study,  the  Team  Resource 
Allocation  Problem  (TRAP).  The  second  section  outlines  the  methods 
followed  in  carrying  out  the  study. 

4 . 1  Experimental  Task 

This  research  is  part  of  a  series  of  experiments  conducted  at  the 
Harry  G.  Armstrong  Aerospace  Medical  Research  Laboratory  (AAMRL)  at 
Wright-Patterson  Air  Force  Base,  Ohio,  under  the  overall  title,  ‘The  Use 
of  Team  Technologies  to  Enhance  Multi-Human  Decision  Making.'  The 
purpose  of  the  research  program  is  to  understand  the  variables  that 
influence  small  group  decision  making  in  an  Air  Force  command  and 
control  setting.  Command  and  control  decision  making,  which  is  closely 


related  to  performance  review  and  monitoring,  is  among  the  tasks  for 
which  empirical  research  concerning  the  usefulness  of  graphic  displays 
is  most  lacking  [JARVBOb],  The  Series  of  experiments  at  AAMRL  uses 
variations  of  a  dynamic  group  choice  task  called  the  Team  Resource 
Allocation  Problem  (TRAP). 

4.1.1  Team  Resource  Allocation  Problem  (TRAP) 

The  TRAP  was  designed  to  simulate  both  the  individual  cognitive 
processes  and  the  small  grou>’  interaction  processes  involved  in  making 
command  and  control  decisions.  It  was  derived  from  the  task  used  by 
Pattipati,  Kleinman,  and  Ephrath  [PATT83)  to  study  dynamia  individual 
decision  making.  The  TRAP  is  intended  to  be  a  general  problem, 
adaptable  to  a  variety  of  research  objectives. 

In  its  current  form,  the  TRAP  is  designed  for  three-person  teams 
who  are  seated  side-by-side  and  allowed  to  communicate  freely.  Figure 

4.1  depicts  the  seating  arrangement.  Each  team  member  has  a  four-button 
control  box,  which  is  illustrated  in  Figure  4.2.  Identical  TRAP 
displays  are  presented  simultaneously  to  each  team  member  via  Silicon 
Graphics  2400  RGB  high-resolution  computer  graphics  work  stations. 
Figure  4.3  portrays  a  TRAP  display  at  a  discrete  point  in  time  as  it 
might  appear  using  the  alphabetic  treatment. 


FIGURE  4.1,  Saatlng  Arrangamant 


FIGURE  4.2.  Control  Box 

4.1.2  TRAP  Evtntl 

Aa  many  it  11  avanta  may  ba  portrayad  at  a  tima,  aach  appaaring  on 
a  aeparata  row  in  tha  TRAP  diaplay.  A  taam  aarna  pointa  by  committing 
ita  raaourcaa  to  an  avant  or  avanta  for  tha  raqulrad  ptrlod  of  tima.  Aa 
brlafad  to  aaoh  taam  during  a  training  aaaaion  (aaa  Appandix  B) ,  thair 
objactlva  for  tha  TRAP  ia 

to  aooumulata  aa  many  pointa  aa  poaaibla  aa  a  taam.  Thia 
maana  diaouaalng  altarnativaa  with  tha  othar  mambara  of  tha 
taam  in  ordar  to  maka  optimum  aalaotion  of  avanta.  Aa  thara 
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will  be  more  events  than  the  team  can  possibly  process, 
combination!  of  «v«nti  should  ba  aalaotad  which  optimize  tiara 
parformanca  and  total  point  aount. 

Evanti  ara  rapraiantad  Symbolically  on  tha  TRAP  diaplay.  Eaoh  naw 
avant  la  assigned  randomly  to  an  availabla  row  in  tha  dlaplayt  ita 
aymbol  appaara  initially  just  to  tha  right  of  column  0  in  ita  assigned 
row,  Tha  avant  aymbola  mova  aoroaa  tha  diaplay  at  a  common ,  abaady  rata 
until  th«y  diaappaar  from  tha  right  aida  of  tha  aoraan. 

4. 1,3. 1  TRAP  Evant  Timing.  -An  avant  aymbol  appaara  on  tha  diaplay  for 
a  total  of  30  tima  unita,  and  taama  ara  required  to  oommib  raaouroaa  to 
an  avant  for  10  tima  unita  to  aarn  ita  pointa.  Undar  tha  high  tima 
abraaa  condition  a  tima  unit  ia  1  aaoond  with  a  naw  avant  aymbol 
appearing  avary  3.73  aaconda;  undar  tha  modarata  tima  atraaa  condition  a 
tima  unit  ia  3  aaconda  with  a  naw  avant  aymbol  appearing  avary  9.40 
aaconda.  Pilot  atudlaa  ravaalad  that  a  3-aacond  tima  unit  allowa  good 
taama  to  approach  maximum  acoradi  aoored  ara  dignif ioantly  lowar  with  a 
1-aacond  tima  unit.  Tha  rataa  at  which  naw  avanta  appaar  provida  for 
tha  poaaibility  of  dlaplaylng  an  avant  on  aaoh  of  tha  11  rowd  at  tha 
aama  tima  (a.g.,  30  tima  unita/11  rowi  •  3.73  aaconda  batwaan  avanta). 

Each  TRAP  trial  undar  tha  high  tima  atraaa  condition  id 
approximately  4  minuted  long;  triala  undar  modarata  time  atraaa  taka 
approximately  8  minuted  each.  Taama  have  to  make  their  decidiond  mora 
quickly  undar  tha  high  tima  atraaa  condition;  avanta  appaar  at  a  faatar 
rata  and  mova  acrodd  tha  diaplay  mora  quickly.  However,  equivalent 
aoorea  ara  poaalble  undar  either  tima  atraaa  condition.  Tha  number  of 
avanta  praaantad  (44  certain  avanta  and  30  uncertain  avanta)  and  tha  mix 
of  point  valued  available  in  a  trial  ara  tha  aama  undar  both  conditlond. 
Becauae  tha  amount  of  tima  that  team  memberd  ara  required  to  commit 
reaouraaa  to  an  avant  la  proportional  to  tha  duration  of  a  time  unit, 
taama  can  theoretically  complete  tha  aama  number  of  avanta  and  aarn  tha 
aamt  acoraa  undar  both  lavala  of  tima  atraaa. 

4. 1.2.3  TRAP  Evant  Typed.  Two  typ*a  of  avanta  ara  repredented  in  tha 
TRAP;  certain  and  uncertain  avanta.  Certain  avanta  ara  repredented  aa 
circled  and  uncertain  avanta  ad  black  rectanglea.  Tha  payoff  for 
committing  raaouroaa  to  aertain  avanta  id  known  and  guaranteed.  Tha 
payofl  for  committing  raaouroaa  to  uncertain  avanta  ia  initially  unknown 
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and,  once  known,  is  not  guaranteed.  (A  specified  probability  exists 
that  no  points  will  be  earned  for  committing  resources  to  uncertain 
events . ) 

4, 1.2.2. 1  Certain  Events.  Table  4.1  identifies  the  possible  point 
values  for  certain  events  and  the  frequency  with  which  each  event  value 
appears  in  a  TRAP  trial.  A  TRAP  trial  presents  a  total  of  44  certain 
events,  distributed  such  that  each  team  member  has  an  equal  opportunity 
to  participate  in  events  of  a  given  value.  For  example,  the  six  1* 
polnt-per-person,  one-person  events  in  eaoh  trial  are  assigned  as  two 
for  eaoh  team  member.  A  small  set  of  events  of  randomly  selected  values 
is  also  presented  at  the  beginning  and  end  of  eaoh  trial;  performance  on 
these  buffer  events  is  not  analysed. 


TABLE  4.1,  Point  Values  for  Certain  Events 


Points 

Per  Person 

Total 

Points 

Frequency 

1 

1 

0 

one- 

Person 

Events 

3 

3 

12 

5 

B 

0 

Two- 

2 

4 

0 

Person 

Events 

4 

8 

0 

Three- 

Person 

Events 

1 

3 

2 

3 

9 

4 

9 

19 

2 

4. 1.2. 2, 2  Uncertain  Events.  Mixed  among  the  44  certain  events 
presented  to  the  participants  are  20  events  of  initially  unknown  value 
(some  'black  boxes'  among  the  coded  circles).  When  a  blaok  box  appears, 
participants  may  get  additional,  though  imperfect,  information  about  the 
event  if  all  three  team  members  commit  their  resources  to  seeking 
Information.  Following  that  commitment  and  a  brief  time  delay  (two  time 
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units),  the  probability  of  payoff,  ‘H'igh  or  *l/ow,  and  the  number  of 
points  possible  for  computing  ths  svtnt  appear  in  reverse  video  on  the 
black  box  (see  Figure  4.3,  Sow  11).  The  team  may  then  choose  to  commit 
or  not  to  commit  resources  to  the  event  as  with  any  other  event, 

During  the  training  session  on  TRAP  with  unoertain  events  (see 
Appendix  0) ,  participants  learn  that 

a.  the  probability  that  committing  resources  to  an  unoertain  event 
will  pay  off  is  either  high  (80  percent)  or  low  (80  percent) 

'  b.  two  possible  point  values  (9  and  81)  are  associated  with  events 
having  a  high  probability  of  payoff  and  two  possible  point  values  (38 
and  84)  are  associated  with  events  having  a  low  probability  of  payoff 

c.  each  of  the  four  possible  types  of  uncertain  events  (H9,  H31, 
L36 ,  L84)  is  equally  likely  to  occur,  and 

d,  considering  the  payoff  probabilities  and  the  time  delay  to  seek 
information,  the  unoertain  events  are,  on  average,  about  equivalent  to 
10-point,  three-person  events  (3  1/3  points  per  person). 

The  payoff  probabilities  and  point  values  used  were  chosen  such 
that  an  optimum  strategy  would  include  two  features: 

a,  seeking  information  about  a  majority  of  the  black  boxoS  and 

b,  committing  resources  to  about  half  of  the  unoertain  events  for 
which  additional  information  is  obtained  (those  for  which  the  expected 
value  exceeds  the  value  of  other  events  available  on  the  display) • 

Scoring  for  the  unoertain  events  varies  from  that  for  other  events. 
A  team  that  commits  resources  to  an  uncertain  event  expects  to  receive 
all  or  none  of  the  possible  points  for  the  event,  depending  on  whether 
or  not  the  event  pays  off.  Therefore,  scores  displayed  to  the  team 
reflect  award  of  all  or  none  of  the  possible  points.  Of  the  five  H9 
events  available  in  a  trial,  for  example,  one  is  randomly  selected  in 
advanoe  to  pay  no  points  should  the  team  commit  resources  to  that  event, 

The  researcher,  however,  is  interested  in  comparing  the  quality  of 
the  decision  making  rather  than  degrees  of  luok  among  the  teams,  Scores 
could  be  distorted,  for  example,  if  a  team  randomly  received  a  large 
number  of  points  for  committing  resources  to  a  few  low  probability 
events  or  if  a  team  fails  to  gain  any  points  despite  committing 
resources  to  a  number  of  high  probability  events,  Therefore,  separate 


scored  are  computed  f or  research  purposes  which  add  the  expected  value 
(the  probability  of  payoff  times  the  possible  point  value)  of  each 
completed  uncertain  event  to  the  accumulated  points  for  the  certain 
events.  Teams  should  elect  to  commit  resources  to  uncertain  events  only 
when  the  events'  expected  values  exceed  the  values  of  other  events  on 
the  display.  Therefore,  scores  based  on  expected  values  more  accurately 
represent  decision  makinf  quality. 

4. 1.2.3  Committing  Resources  to  TRAP  Events.  Some  events  may  be 
handled  by  an  individual  team  member:  handling  other  events  may  require 
a  specified  set  of  two  or  all  three  team  members  to  oommit  their 
resources.  The  black  squares  in  oolumnd  A.  B.  and  C  in  the  TBAP  display 
indicate  which  team  members  must  oommit  resources  to  an  event  to  earn 
its  points.  The  black  square(s)  appear  on  a  given  row  at  the  same  time 
that  an  event  symbol  appears  on  that  row  and  remain  on  the  display  as 
long  as  the  event  Symbol  remains.  On  the  display  depioted  in  Figure 
4.3,  team  member  A  can  handle  the  event  on  row  2  individually  (a  black 
square  appears  in  oolumn  A  of  row  2) ,  but  both  team  members  A  and  C 
would  be  required  to  commit  resources  to  the  event  on  row  6.  No  black 
squares  appear  on  row  3  since  no  event  is  available  on  that  row, 

Each  team  member  controls  a  separate  cursor  which  can  be  moved  up 
or  down  (using  the  up  and  down  buttons  on  the  control  box)  to  any  row  in 
the  member's  column  on  the  display.  The  oursor  (an  asterisk)  is  green 
when  the  team  member  is  available  to  commit  resources  to  an  event  (see 
Figure  4.3,  Column  A,  Row  3).  A  team  member  commits  resources  to  an 
event  by  pressing  the  start  button  on  the  control  box  while  the  oursor 
is  on  the  row  of  a  chosen  event,  at  which  point  the  cursor  burns  red 
(see  Figure  4,3,  Column  B,  Bow  1).  If  additional  members  are  required 
to  commit  resources  to  the  same  event,  the  black  square  in  the  first 
member's  column  turns  pink  indicating  ‘waiting*  status  (see  Figure  4,3, 
Column  B,  Bow  l).  The  cursors,  even  while  they  are  red,  may  be  moved  to 
other  rows  in  preparation  for  other  events  (s.g.,  team  member  0  has 
moved  the  oursor  from  Bow  7  to  Row  8  in  Figure  4.3).  The  start  button 
will  not  work,  however,  until  the  previous  commitment  is  complete,  A 
member  may  abort  a  commitment  to  an  event  by  pressing  the  reset  button 
on  the  control  box. 


4. 1.2. 3.1  Opportunity  Window.  In  order  to  commit  resources  to  an^evant 
for  a  full  10  time  units  before  tha  svsnt  leaves  th«  screen,  •  teem  must 
start  the  event  before  it  reaohes  the  end  of  the  opportunity  window, 
The  end  of  the  window  is  indicated  by  a  vertioal  line  placed  such  that 
events  reaching  the  line  will  remain  on  the  display  for  10  additional 
time  units  (see  Figure  4,3).  If  a  team  has  not  Started  an  event  by  the 
time  it  reaches  the  end  of  the  window,  the  team  won't  be  able  to  commit 
resources  to  the  event  for  a  sufficient  amount  of  time  and  won't  be  able 
to  earn  any  points  lor  that  event. 

When  all  required  members  have  committed  resources  to  an  event,  the 
black  square(s)  turn  yellow  (see  Figure  4,3,  Column  0,  Row  7) .  In 
addition,  a  horisontal  black  bar  appears  in  that  event's  row  (see  Figure 
4.3,  Row7) .  The  bar  starts  where  the  event  symbol  was  when  all  required 
resources  were  committed,  It  extends  to  the  right  for  the  distance  the 
event  symbol  will  move  in  10  time  units,  the  resouroe  commitment  time 
required  to  earn  points  for  the  event,  The  bar  remains  stationary,  and 
the  event  symbol  moves  through  the  bar.  When  the  event  symbol  moves  out 
of  the  bar,  points  for  completing  the  event  are  automatically  added  to 
the  accumulated  points  table  in  the  lower  left  corner  of  the  display, 
and  the  aursors  for  team  members  whose  resources  were  committed  to  the 
event  turn  green.  At  the  same  time,  the  event  symbol,  the  yellow 
squares  in  aolumns  A,  B,  and  0  on  the  same  row,  and  the  black  bar 
disappear. 

4. 1.2. 3. 2  Resource  Status.  The  table  in  the  lower  right  portion  of  the 
display  (see  Figure  4,3)  provides  details  on  the  status  of  team  member 
resources.  A  black  square  in  the  ‘Free*  row  indicates  that  a  team 
member's  resources  are  available  (see  Column  A  of  the  table  in  Figure 

4.3) .  A  blinking  pink  square  in  the  ‘Wait*  row  indicates  that  a  team 
member's  resources  are  committed  but  nonproductive  until  other  member(s) 
commit  resources  to  the  same  event  (see  Column  B  of  the  table  in  Figure 

4.3) ,  A  digit  in  the  ‘Work*  row  indicates  the  number  of  time  units 
remaining  before  a  team  member's  resources  will  be  freed  from  a  current 
commitment  (see  Column  C  of  the  table  in  Figure  4,3), 

4. 1.2. 3. 3  ' Model*  Resource  Commitment,  A  computer  model  that  uses 
'correct*  logic  in  performing  the  TRAP  has  been  developed  as  a  basis  for 
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comparison  with  performance  by  human  teams.  The  computer  model  compares 
the  values  of  all  alternatives  and  commits  resources  to  the  highest 
value  event  or  combination  of  events  available  at  a  point  in  time,  It 
repeats  its  assessment  and  resourae  commitment  cycle  each  time  its 
resources  are  freed  throughout  a  trial,  (Note  that  the  model 
suboptimises  on  the  events  available  at  a  given  point  In  time  rather 
than  optimising  over  an  entire  trial.) 

For  research  purposes,  overall  team  scores  are  stated  as  ratios  of 
the  points  earned  by  the  team  to  the  points  amassed  by  the  model,  Team 
scores  reflecting  proportions  of  eaoh  event  type  completed  are  also 
aompared  to  model  results.  Relating  team  scores  to  'best  possible* 
scores  helps  to  alleviate  any  bias  associated  with  variations  in  the 
difficulty  of  the  experimental  trials  (trial  difficulty  varies  due  to 
the  randomised  sequencing  of  events). 

4.1,3  Presentation  Treatments 

One  objective  of  this  research  was  to  examine  the  effeots  of 
different  Information  presentation  forms  on  group  decision  quality  and 
perceived  individual  workload.  That  objective  was  met  by  measuring  the 
impact  of  variations  in  the  features  used  to  represent  certain  events. 
This  subsection  explains  the  presentation  treatments  used. 

Recent  researoh  on  Group  Decision  Support  System  design  suggests 
that  group  performance  should  be  enhanced  when  features  of  the  group 

decision  support  system  match  the  requirements  of  the  decision  task 

* 

IDESA00] .  For  intellective  tasks  such  as  TRAP,  DeSanotls  and  Oallupa 
suggest  that  the  Qroup  Decision  Support  System  should  provide  assistance 
in  identifying  the  correct  alternative  [DESABO].  Features  used  to 
represent  events  in  the  TRAP,  therefore.  Should  aid  in  identifying  the 
moat  benefloial  events. 

This  researoh  examined  a  set  of  presentation  features  to  determine 
which  best  support  identifying  high~value  TRAP  events.  Three 
alternative  forms  of  the  TRAP  have  been  developed  to  allow  between-group 
comparisons  among  presentation  treatments,  each  using  a  different  event 
coding  scheme.  The  three  presentation  treatments  use  either  a  single* 
featured  coding  scheme  (a  color  scheme  or  an  alphabetic  scheme) 


or  a  conjunctiva  achama,  Tha  conjunctiva  Bahama  oombinaa  tha  cojor  and 
alphabatie  aohamaa, 

Tabla  4.3  praaanta  tha  coding  aohamaa  uaad  in  tha  study.  In  tha 
color  achama,  avanta  appaar  aa  color-f illad  oiroiaai  in  tha  alphabatla 
achama,  avanta  appaar  aa  yallow»( Iliad  oirolaa  with  black  capital 
lattara  oantarad  within  tha  oirolaa |  it,  tha  conjunctiva  aohama,  avanta 
appaar  aa  rad-  or  blua-fillad  ,  oirolaa  with  black  capital  lattara. 
Tabla  4,3  llluatrataa  how  aach  of  tha  poaalbla  point  valuaa  la 
rapraacntad  in  TRAP  avant  symbols, 

TABLE  4.2.  Oodin|  Sehamaa  for  TRAP 


(Points  par  Parson) 
13  3  4  5 


Tha  color  achama  uaad  fits  with  astabllshad  guidelines  (or  color 
displays.  Tha  numbar  o(  oolora  uaad  ia  limitad  IDAVIB3,  IVE8B3],  and 
tha  colors  ara  ordarad  by  thair  appaaranoa  in  tha  rainbow  (IVE8B31,  Tha 
ordaring  also  conforms  to  'well-established  habits  or  population 
staraotypas'  C D AVI 83 ,  p,  118]  in  that  rad  is  uaad  to  idantify  tha  moat 
important  (highaat  valua)  avanta,  Thara  ia  also  amplrlcal  praoadanoa 
(or  tha  color  aohama  uaad;  Christ  uaad  a  Six-laval  color  coda  (purpla, 
blua,  green,  yillow,  orange,  and  rad)  in  hia  atudiaa  (0HR163), 

Tha  alphabatie  achama  uaad  dilfara  from  tha  act  uaad  by  Ohriati  ha 
uaad  tha  lattara  C,  H,  K,  N,  P,  and  8  to  minlmisa  eonfusabillty  among 
tha, shapes  of  tha  lattara  (CHR1B31,  Tha  ordarad  sat  (A •  B,  C,  D,  K)  was 
choaan  instaad  to  conform  to  population  staraotypas.  Sinoa  tha 


participant!  were  college  students  familiar  with  letters  *a  .course 
grades,  A  was  ohossn  to  rapraaant  tha  highest  value  avants, 


TABLE  4.3,  Event  Symbols  for  Gsrtain  Evanti 


Point* 

Par  Total 

Color  Alphabatio  Conjunctiva  Paraon  Polnta 


One- 

Person 

Events 

blue 

I 

blue 

0 

1 

B 

yellow 

0 

blue 

A 

jnj 

red 

A 

red 

A 

S 

0 

Two- 

Person 

Events 

green 

D 

blue 

B 

s 

4 

orange 

B 

red 

B 

3 

fl 

Three- 

Person 

blue 

E 

blue 

C 

i 

yellow 

0 

blue 

A 

3 

D 

Events 

red 

A 

red 

A 

0 

19 

Tha  conjunotlva  coda  used  la  a  simple  combination  of  tha  oolor  and 
alphabetic  coda*.  Rad  and  blua  ware  ehoaan  from  tho  oolor  scheme  (or 
thalr  distinctiveness,  and  At  B,  and  0  ware  chosen  to  paintaln  tha  Sanaa 
of  ordering  astabllahad  in  the  alphabatio  schema.  Six  color-latter 
combinations  are  possible  in  this  conjunctive  schema;  tha  rsd-C 
combination  was  arbitrarily  dropped  to  provide  tha  required  five-level 
coding  schema. 

4.1.4  Heuristic!  Trsatmtnt 

In  addition  to  aKaminlitg  tha  affects  of  different  information 
presentation  forms  on  team  performance ,  an  objective  of  this  research 
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was  bo  determine  il  task  performance  could  be  improved  by  providing  and 
encouraging  discussion  of  decision  heuristios.  Between-group 
comparisons  were  made  to  detect  performance  differences  with  and  without 
the  provision  and  disoussion  of  decision  heuristics.  This  subsection 
provides  details  concerning  the  heuristios  treatment, 

4. 1.4.1  Basis  for  Heuristics.  The  decision  heuristics  were  built  based 
on  information  gained  from  three  louroeet  the  computer  model  against 
which  team  performance  is  compared)  the  researohers  who  developed  the 
TRAP,  and  participants  in  TRAP  pilot  studies. 

Comparing  the  oholoes  made  by  participants  in  a  pilot  study  to 
those  made  by  the  computer  model  over  a  series  of  33  trials  suggested 
several  heuristics.  For  example,  pilot  study  participants  tended  to 
work  significantly  fewer  high  value  two-person  events  than  the  model 
did,  suggesting  that  a  heuristic  to  look  for  suoh  events  would  be 
useful.  The  model  also  rarely  works  on  the  lowest  value  events,  so  a 
heuristic  to  ignore  such  events  was  included. 

The  TRAP  developers  also  provided  decision  rules  based  on  their 
knowledge  of  task  design  features  and  on  their  repeated  observations  of 
teams  performing  the  TRAP.  For  example,  they  suggested  the  decision 
rule,  'Keep  the  team  synchronised;  all  three  team  members  should  commit 
their  resources  to  events  at  the  same  time.1  Their  suggestion  comes 
from  observations  that  teams  sometimes  waste  time  or  miss  opportunities 
to  complete  high  value  two-  or  three-person  events  while  waiting  for 
individual  team  members  to  release  previously  committed  resources. 
Since  the  amount  of  time  for  which  resources  must  be  committed  is 
identical  for  all  events,  teams  that  commit  their  resources  in  unison 
will  also  have  their  resources  free  for  a  Subsequent  commitment  all  at 
the  same  time, 

Finally,  participants  in  a  pilot  Study  indicated  through  their  task 
performance  and  through  their  responses  to  questionnaires  and  interviews 
that  teams  vary  slgnif ioantly  in  their  handling  of  uncertain  events.  In 
partioular,  their  strategies  rarely  include  using  expected  value  to 
guide  decisions  about  uncertain  events.  Since  using  expected  value  can 
Improve  performance  on  the  TRAP,  a  heuristic  based  on  the  expected  value 
of  the  uncertain  events  was  included.  Additional  pilot  testing  also 
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provided  performance  data  and  interview  material  used  to  line  tune  the 
wording  and  sequencing  of  the  choaen  heuriatioa. 

4. 1.4. 2  Choaen  Heuristics.  The  set  of  decision  rules  chosen  represents 
a  balance  between  comprehensiveness  and  retalnability .  The  set  is 
oomprehensive  enough  to  improve  performance  but  limited  enough  to  avoid 
overloading  human  information  processing  capacity,  Teams  included  ir, 
the  heuriatioa  treatment  received  an  introduction  to  the  heuristics  and 
their  rationale  and  were  encouraged  to  discuss  the  heuristics  during  a 
training  session  (see  Appendix  D.)  Stated  in  terms  appropriate  to  the 
alphabetic  treatment,  the  deolsion  heuristics  used  are  as  follows: 

HEURISTIC  *1:  In  addition  to  S-person  A  events,  look  tor  a  2- 
person  B  event,  especially  with  a  1-person  A  event. 

HEUR1STI0  *2:  Next,  check  as  many  uncertain  events  as  you  can, 
Immediately  take  L84  or  H21|  ignore  L38  and  HO. 

HEURISTIC  «3;  Ignore  D  and  S  events. 

HEURISTIC  *4:  Keep  the  team  synchronised,  All  three  team  members 
should  start  an  event  or  combination  of  events  at  the  same  time, 

4.2  Research  Procedure 

In  describing  the  TRAP,  the  previous  section  introduced  Some  of  the 
procedures  used  to  carry  out  this  research,  This  seotlon  presents  the 
research  procedure  in  greater  detail, 

4.2.1  gatUalEMli 

The  participants  in  the  experiment  were  72  college  students  in  the 
Dayton,  Ohio  area.  They  were  reoruited,  scheduled,  and  paid  190  each  by 
Systems  Research  Laboratories,  Inc,,  an  agency  that  oontracted  to 
provide  support  for  research  programs  at  Wright-Patterson  Air  Force 
Base,  All  had  at  least  30/40  corrected  vision  (20/70  vision  is  adequate 
to  perform  the  TRAP).  The  participants  were  also  screened  for  normal 
color  vision  using  lahihara  platos,  Three  individuals  whose  initial 
screening  suggested  color  vision  limitations  were  assigned  to  the 
alphabetic  treatment  and  then  underwent  additional  screening  during  the 
training  session  to  assure  that  they  oould  properly  distinguish  features 
on  the  TRAP  display,  In  addition,  no  participants  who  wers  involved  in 
previous  TRAP  studies  were  eligible  to  participate  again, 


No  restrictions  were  placed  on  team  composition,  Participants  were 
assigned  to  teams  based  on  their  availability  (olass  and  work  schedules, 
transportation,  etc.).  Eight  teams  were  assigned  randomly  to  each  of 
the  presentation  treatment  groups.  Among  each  of  the  B-team  groups, 
half  were  randomly  assigned  to  each  of  the  decision  strategy  treatments, 
4.2.2  Experimental  Procedure 

The  experiment  was  conducted  in  an  established  experimental 
facility  within  the  Armstrong  Aeromedlcal  Research  Laboratory,  Wright- 
Patterson  Air  Foroe  Base,  Ohio.  Teams  consisted  of  three  participants 
each  who  were  seated  side  by  side  (positions  were  randomly  assigned). 
Each  participant  had  a  CRT  and  a  control  box  for  performing  the  TRAP. 
Participants  wore  headsets  with  lip  microphones  for  recording  team 
communication, 

Each  team  participated  in  four  sessions,  each  conducted  on  a 
separate  day.  The  first  session  covered  administrative  requirements 
(consent  forms,  visual  screening,  and  the  first  data  collection  phase  of 
the  Subjective  Workload  Assessment  Technique  (see  Appendix  A)). 
Participants  were  trained  during  the  second  session,  and  the 
experimental  trials  were  run  during  the  last  two  sessions. 

Beaause  the  TRAP  is  very  complex,  training  and  praatlce  prior  to 
experimental  participation  is  required.  A  training  session  of 
approximately  two  hours  familiarized  participants  with  the  TRAP,  first 
without  uncertain  events  and  then  with  uncertain  events,  Each  team  was 
trained  using  versions  of  the  TRAP  appropriate  to  the  presentation 
treatment  group  to  which  the  team  was  assigned.  Those  teams  assigned  to 
the  heuristics  treatment  groups  received  instruction  on  the  heuristics 
immediately  following  training  on  the  TRAP  with  uncertain  events  and 
were  allowed  time  to  discuss  their  docision  str* .  They  were  alio 
allowed  to  retain  printed  copies  of  the  heurlstiol  for  use  in  the  test 
sessions.  Participants  were  tested  during  the  training  session  to 
assure  that  they  understood  how  event  point  values  were  determined  and, 
for  those  teams  who  received  heuristics,  that  they  understood  how  to 
apply  the  heuristics.  Eech  team  also  completed  two  practice  trials 
without  uncertain  events  and  an  additional  four  practice  trials 


with  uncertain  events  with  free  opportunity  to  ask  questions  ^and  to 
communicate  with  one  another. 

Following  the  training  session,  each  team  completed  two 

experimental  sessions,  scheduled  on  separate  days.  Each  experimental 
session  consisted  of  two  test  sessionsi  and  eaoh  test  session  consisted 
of  eight  trials.  At  the  end  of  each  trial,  the  participants  provided 

data  for  phase  two  of  the  Subjective  Workload  Assessment  Teohnique  (see 
Appendix  A),  All  experimental  sessions  were  audio/videotaped,  and  tapes 
of  the  best  three  and  poorest  three  teams  (baled  on  average  deoilion 
quality  scores)  were  reviewed  to  determine  the  extent  to  whioh  teams 

discussed  their  task  performance  strategies  and  explicitly  applied  the 

decision  heuristics.  The  analysis  of  the  tapes  provided  subjective 
explanatory  material  rather  than  statistically  analysed  results,  It 
helped  to  identify  differences  among  the  team  in  applying  the  heuristics 
as  well  as  differences  among  the  suggested  heuristios  in  their  rates  of 
application.  It  also  helped  to  detest  whether  teams  that  did  not 
receive  suggested  heuristios  developed  similar  or  alternate  heuristios 
of  their  own. 

In  greater  detail,  the  experimental  procedure  consisted  of  the 
following  steps  (copies  of  materials  used  for  the  oolor  coding  scheme  in 
the  experiment  are  included  as  appendices;  equivalent  materials  were 
used  for  other  treatment  groups); 

1.  First  session  (approximately  one  hour):  Screened  participants 

and  administered  preliminary  materials. 

a.  Researchers  in  the  Visual  Display  Systems  Branch, 
Armstrong  Laboratory,  administered  color  vision  screening 
using  Ishlhara  plates  under  the  prescribed  lighting 
conditions.  The  experimenter  administered  visual  acuity 
screening  using  Snellen  charts  (participants  were 
screened  for  30/40  corrected  vision;  the  TRAP  display 
required  20/70  vision). 

b.  Participants  read  and  signed  two  consent  forms  (Appendix 
E) .  The  forms  introduced  the  research  objectives  and 
outlined  the  terms  of  the  participants'  agreement  to 
serve  as  team  members  for  the  experiment.  Copies  of  the 
forms  were  made  available  to  the  participants. 

c.  The  experimenter  introduced  the  Subjective  Workload 
Assessment  Technique  using  a  standard  script  (Appendix  F) 


and  provided  curd  docks  and  SWAT  luminary  handout!  1  or  uii 
during  thi  aorting  prootdun.  Participants  completed  ths 
SWAT  sorting  procedure.  (SWAT  sort  data  has  been  shown 
to  ba  stabla  for  as  long  as  a  yaar  [HARR87],  so 
participants  for  whoa  raoant  SWAT  sort  data  was  available 
wars  not  required  to  rapaat  tha  sorting  procedure.) 

2.  Second  session  (approximately  two  hours):  Trained 

participants. 

a.  Tha  participants  ware  randomly  assigned  to  seating 
positions  to  be  retained  for  all  remaining  sessions. 

b.  The  experimenter  read  from  a  standard  script  (Appendix  B) 

providing  instruction  on  performing  tha  baslo  TRAP 
(without  uncertain  events).  The  instructions  were 
tailored  to  the  presentation  format  assigned  to  tha  team. 
At  tha  appropriate  point  in  the  instruction,  the 

participants  reviewed  a  table  of  TRAP  point  values 
(attached  to  Appendix  B)  and  then  took  a  test  on  their 
understanding  of  the  point  values  (Appendix  Q) . 
Individual  feedback  and  testing  was  repeated  as  required 
until  all  participants  completed  the  test  perfectly,  A 
slow  speed  TRAP  demonstration  trial  with  commentary  was 
included  in  the  instruction  period  following  point  value 
testing. 

o.  The  participants  performed  two  practice  trials  (one  at 
moderate  and  one  at  high  time  stress  levels),  Following 
each  trial,  each  participant  responded  to  three  workload 
questions  presented  on  the  CRT.  The  experimenter 
answered  any  questions  that  oame  up  during  the  practice 
trials. 

d.  The  experimenter  read  from  a  standard  script  (Appendix  C) 

providing  instruction  on  performing  the  TRAP  with 
uncertain  events,  The  participants  read  along  from 

copies  of  the  instructions  and  interacted  with  a  slow 
speed  TRAP  demonstration  trial  including  uncertain 
events. 

e.  For  teams  assigned  to  the  heuristics  treat-men'*  groups, 

the  experimenter  read  from  a  standard  sorlpt  (Appendix  D) 
providing  instruction  on  performing  the  TRAP  using  the 
heuristics.  The  instructions  were  tailored  to  the 

presentation  format  assigned  to  the  team.  The 

participants  read  along  from  copies  of  the  instructions. 
The  participants  then  received  a  summary  of  the 

heuristioi  (attached  to  Appendix  D)  to  retain  during  the 
remainder  of  the  session  and  were  encouraged  to  discuss 
the  heuristios  and  their  decision  strategy  as  a  team. 
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a. 


f.  For  team*  assigned  to  the  heuristics  treatment  groups, 
the  participant*  took  a  teat  on  thtlr  understanding  of 
the  heuristic*  (Appendix  H) .  Individual  feedback  and 
testing  wa*  repeated  aa  roquirad  until  all  participant* 
completed  tha  taat  perfectly. 

|.  Tha  participant!  parformad  four  praotioa  triala  (two  at 
modarata  and  two  at  high  time  atraaa  lavala;  tha  aaquano* 
of  modarata  and  high  tima  atraaa  triala  was  moderate- 
high-moderate-high)  .  Following  aach  trial,  aaoh 

partioipant  raapondad  to  thraa  workload  quaatlona 

praaantad  on  tha  OUT, 

3,  Third  and  fourth  aaaaiona  (approximately  two  houra  aaoh) : 

experimental  aaaaiona. 

a.  Tha  participant!  ware  allowed  to  review  a  table  of  TRAP 
point  value!  and  then  took  a  taat  on  their  undaratanding 
of  tha  point  value*.  Individual  feedback  and  taating 
ware  repeated  aa  required  until  all  participant* 
oomplatad  tha  taat  perfectly. 

b.  For  team*  aaaignad  to  tha  hourlatioa  treatment  group*, 
the  participant*  war*  allowed  to  review  and  retain  a 
aummary  of  tha  hauriatlc*  and  took  a  teat  on  their 
undaratanding  of  the  haurlatioa,  Individual  feedback  and 
taating  waa  repeated  aa  required  until  all  participant* 
completed  the  teat  perfectly. 

a.  The  participant*  performed  two  teat  aeaaiona  oonaiating 
of  eight  trial*  each  (four  at  moderate  and  four  at  high 
time  atraaa  level* j  the  aequenc*  of  moderate  and  high 
time  atraaa  trial*  waa  randomlaed  in  block*  of  four 
trial*).  Following  each  trial,  each  partioipant 
raapondad  to  three  workload  queationa  preaentad  on  the 
CRT.  A  break  waa  allowed  between  the  two  teat  aaaaiona. 

4 .3  Summary  of  Beaearch  Environment  and  Method 

Thla  atudy  involved  a  laboratory  experiment  uaing  a  dynamic  group 

choice  taak  (TRAP)  aa  the  experimental  teak.  Subaaquent  to  screening 

and  training,  each  team  of  three  deoialon  maker*  completed  a  total  of  33 

trial*,  arranged  in  4  teat  aaaaiona  of  8  trial*  each.  Half  of  the 

trial*  were  preaentad  at  a  faat  rate  of  apeed  and  halt  at  a  more 

moderate  apeed.  Eaah  TRAP  trial  preaentad  a  aequenc*  of  64  certain  and 

uncertain  aventa,  The  team*  had  inaufficient  resource*  to  respond  to 

every  event,  ao  they  had  to  ohooa*  among  the  available  events.  Coding 

schemes  for  presenting  the  event*  varied  between  team*  aa  did  the 

presence  or  absence  of  decision  *  juristic*  provided  by  the  researcher. 

08 


The  research  method  outlined  in  this  chapter  provided  the  raephanilm 
by  which  to  capture  information  on  the  effeats  of  information 
presentation  form  and  decision  heuristics  on  group  decision  making  in  a 
dynamio,  uncertain  environment.  The  next  chapter  analyses  the  results 
of  the  experiment. 
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5,0  ANALYSIS  OF  RESULTS 

This  chapter  presents  the  statistical  analysis  of  the  results  of 
the  experiment.  (Cell  means  and  variances  and  the  raw  data  are  included 
in  Appendix  I.)  The  first  Section  reviews  the  major  results  of  a 
related  study.  The  second  section  defines  the  primary  model  used  to 
evaluate  the  results  of  this  study,  and  the  third  section  presents  the 
findings  using  the  model.  The  model  used  to  analyse  findings  about 
Information  seeking  behavior  and  the  findings  based  on  the  model  are 
presented  in  the  fourth  and  fifth  sections.  Similarly,  the  model  and 
findings  concerning  resource  commitments  to  uncertain  events  appear  in 
the  sixth  and  seventh  aeotlone.  A  major  section  then  relates  the 
findings  to  the  experimental  hypotheses  presented  in  Chapter  3.  The 
chapter  concludes  with  a  summary  of  the  major  results  of  this  study. 

5. 1  Related  Results 

In  a  related  study  run  concurrently  with  this  experiment,  '.Kimble, 
McNeese,  and  Goodyear  examined  the  effects  of  emergent  leadership  on 
TRAP  performance.  They  determined  that  a  number  of  variables  (age, 
college  year  completed,  sex  composition  of  team,  video  game  experience, 
and  leadership  measured  in  terms  of  relative  frequency  and  duration  of 
talking)  had  no  effect  on  TRAP  performance,  One  variable,  experience 
with  using  computers,  was  correlated  with  performance  (r  ■  .84,  p  < 
.002)  [KZMB87  3  .  Random  assignment  of  participants  to  teams,  however, 
resulted  in  an  even  distribution  of  experienced  computer  users  among  the 
treatment  groups  studied  in  this  experiment,  so  the  results  reported 
here  are  not  confounded  with  the  computer  use  variable. 

5.2  Primary  Experimental  Model 

The  results  of  the  experiment  were  examined  using  the  SAS  ANOVA 
procedure.  Except  for  the  analyses  of  information  seeking  and  resource 
commitment  behaviors  related  to  uncertain  events  (models  to  be  presented 
in  later  sections),  the  model  used  for  the  statistical  analysis  was 

Yijkimn  *  U.  ♦  timei  ♦  SesSj  4  form*  +  heuri  ♦  teamm<m>  4 
(time«sess)u  ♦  (time«f  orm)  tx  *  (time*heur) » i  ♦  ( time » team)  mi  u\  >  + 
(sess»form) jk  4  (sessftheur) j i  ♦  (sees » team) hi >  ♦  (form"heur)ui  * 
1  time»sess*form)  Mh  ♦  (time»sess»heur)  ui  ♦  (time»form«heur)  no  ♦ 
(sess«  form»heur)  4  (time«sess«f orm»heur)  uni  4  entumni 


where 


*Y'  represents  a  dependent  variable  measure  (overall  TRAP  score, 
proportions  of  events  completed,  SWAT  ratings,  etc.) 

‘time'  is  the  time  stress  variable  (high  or  moderate  time  stress) 
‘seas'  is  the  practice  variable  (session  1,  2,  3,  or  4) 

'form*  is  the  information  presentation  form  variable  (color, 
alphabetic,  or  conjunctive  coding  scheme) 

'heur'  is  the  suggested  decision  strategy  variable  (with  or  without 
heuristics) 

'team'  is  a  variable  that  takes  into  account  any  differences  among 
the  three  teams  assigned  to  each  combination  of  treatments  (any  given 
team  was  assigned  to  only  one  combination  of  a  form  treatment  and  a 
heuristics  treatment).  Sinoe  team  differences  are  accounted  for  in  the 
model,  the  results  related  to  other  variables  can  be  generalised  for 
oth«r  teams  with  more  confidence  than  if  team  differences  were  not 
Included.  '  *  * 

Table  0.1  provides  the  degrees  of  freedom  and  expected  mean  squares 
for  the  model.  A  three-way  interaction  term,  (timeMessoteam)  ijmiwn , 
was  pooled  into  the  error  term  Sinoe  no  degrees  of  freedom  remained  for 
the  error  term.  As  a  result,  no  tests  of  team  effects  or  of  the  effects 
of  interactions  between  the  team  variable  and  other  variables  were 
provided  by  the  model,  but  team  effeots  were  not  a  focus  of  this  study. 

Tests  of  the  assumptions  on  whloh  the  ANOVA  model  is  based 
demonstrated  that  the  model  is  applicable  to  the  results  of  this  study. 
Specifically,  within  commonly  acaepted  limits,  all  observations  have  the 
same  variance  and  the  error  terms  are  normally  distributed. 

5,3  Statistical  Analysis  Using  Primary  Experimental  Model 

The  primary  experimental  model  was  used  to  analyse  findings  for 
seven  outcome  variables.  Eaoh  of  the  following  subsections  presents  the 
analysis  for  one  of  the  variables. 

5.3.1  Overall  Performance 

The  variable  used  to  measure  overall  performance  was  a  team  score 
stated  as  a  percentage  of  the  computer  model's  score  for  the  same  trial. 
Table  9.2  is  the  ANOVA  table  for  the  overall  performance  variable. 
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TABLE  9.1  Primary  ANOVA  Model 


Source  DF  Expected  Meen  Square 


time 

1 

9flXaia  ♦  4c»*a  +  o* 

sees 

4BXbja/3  ♦  20k«a  ♦  o* 

form 

04XCk*/2  +  80**  ♦  Oa 

hour 

1 

OOIdi*  ♦  8c.a  ♦  c* 

team(form»heur) 

18 

Bc«a  ♦  o* 

sess«time 

24XZ(ab) * j*/3  ♦  o.k.a  ♦  ca 

form* time 

32XZ(ac) »ka/2  ♦  4o»**  ♦  ca 

heuretime 

1 

48XX(ad) ii*  ♦  4c..a  ♦  o* 

time* team (f orra*heur) 

18 

♦  c# 

form«sess 

0 

lOXX(bo) jmVO  ♦  2o*.a  ♦  era 

heuMsess 

3 

24XZ(bd) j ia/3  ♦  2cw«a  ♦  a* 

sess«team(form*heur) 

84 

In*1  ♦  •* 

formoheur 

2 

32XX(od) ma/2  ♦  8c.a  ♦  oa  > 

timeNsessRform 

8 

BXXX(abo)  t4ka/0  ♦  o.w.a  ♦  o'a  * 

time«sess»heur 

3 

12XXX(abd) tj »a/3  4  o*k.#  ♦  oa 

84 

0*k«a  ♦  oa 

time*form*heur 

2 

lOXXX(acd) mta/2  ♦  4c..a  +  oa 

sess»form»heur 

6 

8XXX(bod) jm*/0  ♦  Bo*."  +  oa 

time»sess»f orm*heur 

6 

4XXXX(abcd) tikta/8  ♦  <ua.a  ♦  oa 

error 

0 

oa 

There  were  no  statistically  il|niflo«nt  three-way  or  higher  order 

interaction  effects.  There  were  alao  no  atatletioally  significant  two- 

* 

way  interaotion  effects  (or  sessetime  or  formeheur.  Using  the  mean 
iquare  for  time»team(form»heur)  ai  an  error  term,  there  wai  no 
statistically  eignifioant  effect  for  formHime  (p  •  .8013),  but  there 
wad  an  effect  for  heuritlme  (p  ■  .0300).  Uaing  the  mean  aquare  for 
sess»taam(form»heur)  ad  an  error  term,  there  were  no  statistically 
significant  affects  for  formesess  (p  *  .9112)  or  for  heurnsoss  (p  * 

.0189).  Since  the  only  significant  interaction  effect  (heur'time)  was 
ordinal  (see  section  9.10.1  below),  all  the  main  effects  were  analysed. 
Using  the  mean  square  for  teamtf orm»heur)  as  an  error  term,  there  was  no 
statistically  significant  effect  for  form  (p  *  .6330).  but  there  was  an 


effect  for  heur  (p  *  .0092) •  Using  the  mean  square  for 
timexteam(formxheur)  as  an  error  term,  thsrs  was  an  effect  lor  time  (p  ■ 
.0001).  and,  using  the  mean  square  lor  sess*team(f ormxheur)  as  an  error 
term,  there  was  an  effect  for  sees  (p  ■  .0003). 


TABLE  6,2  AMOVA  Results,  Overall  Performance 


Source 

DF 

Sum  of  Squares 

Mean  Square 

Model 

137 

13392.97364298 

97.79893170 

Error 

94 

990,10228736 

18,33922794 

Corrected  Total 

191 

14383,07992994 

Source 

DF 

8um  of  Squares  F 

Value  Pr  >  F 

time 

1 

7278.899129 

382,63 

0.0001 

sees 

612.929669 

7.31 

0.0003 

form 

100.921266 

0,47 

0.6336 

heur 

1 

917.202439 

8.91 

0.0092 

team( f ormxheur) 

18 

1939,782624 

(no 

test) 

sessxtime 

3 

19.466077 

0,28 

0.8388 

form* time 

2 

8,039733 

0.22 

0.8013 

heurx time 

1 

109.640107 

9.99 

0.0300 

timexteam(formxheur) 

18 

342.907240 

(no 

test) 

form»sess 

■  6 

67,449408 

0.34 

0.9112 

heurxsess 

3 

13.964043 

0,17 

0.91B9 

sess* team(form«heur) 

94 

1909.826241 

(no 

test) 

formxheur 

2 

148.144004 

0.69 

0.9196 

timexsessxform 

6 

23,699976 

0.43 

0.7331 

timexsessxheur 

3 

63,996863 

0.76 

0.6048 

timexf ormxheur 

2 

71,666409 

1.88 

0.1813 

sessxf ormxheur 

6 

23,899419 

0.14 

0.9898 

time* sees* formxheur 

6 

139.182191 

1.27 

0.2889 

The  overall  performance  score  was  expressed  as  a  team's  percentage 
of  the  computer  model's  score  to  account  for  potential  differences  in 
difficulty  among  the  trials.  Raw  team  Scores  were  also  analysed  to 
provide  an  indication  as  to  the  need  for  standardising  team  scores  by 
the  model  scores,  Table  9.3  presents  the  ANOVA  results  for  raw  team 
scores, 

There  were  no  statistically  Significant  three-way  or  higher  order 
interaction  effects.  There  were  also  no  statistically  significant  two- 
way  interaction  effects  for  sessxtime  or  form»heur,  Using  the  mean 
square  lor  time«team(formxheur)  as  an  error  term,  there  was  no 
statistically  significant  effect  for  form»time  (p  ■  .6920),  but  there 
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was  an  effect  for  heurstime  (p  ■  ,0139).  Using  the  mean  square  for 
sess*team(form»heur)  u  an  error  term,  thers  war*  no  statistically 
significant  affaots  for  formseeee  (p  ■  .6076)  or  for  heurseees  (p  ■ 
,9299),  Sinca  the  only  significant  interaction  effect  (heurstime)  was 
ordinal  (sat  section  5.16.1  below),  all  tha  main  elfects  ware  analysed. 
Using  tha  mean  square  for  teamtf ormsheur)  as  an  error  term,  there  was  no 
statistically  significant  effect  for  form  (p  *  .4919),  but  there  was  an 
effect  for  heur  (p  ■  .0123).  Using  the  moan  square  for 
timesteam(formsheur)  as  an  error  term,  there  was  an  effect  for  time  (p  • 
.0001),  and,  using  the  mean  square  for  sesssteemtf ormsheur)  as  an  error 
term,  there  was  an  affect  for  sees  (p  *  .0001). 


TABLE  6.3  AN0VA  Table,  Team  Score 


Source 

DP 

Sum  of  Squares 

Moan  Square 

Model 

137 

67161.23726260 

417.2363064 

Error 

64 

3962.00742188 

73.3705078 

Corrected  Total 

191 

61123.24467448 

Source 

DF 

Sum  of  Squares  F 

Value  Pr  >  F 

time 

rr 

32148.394805 

419.98 

0.0001 

sess 

2612.563893 

9.94 

0.0001 

form 

706.032214 

0.74 

0.49)9 

heur 

•  1 

3699.979805 

7.75 

0.0123 

teamif ormsheur) 

18 

8596.036641 

(no 

test) 

lass* time 

248.032747 

1.13 

0.3464 

f  ormHime 

66.908203 

0.44 

0.6528 

heurstime 

1 

568.735430 

7.43 

0.0139 

time  steam (forms heur) 

18 

1377.848616 

(no 

test) 

f ormssess 

6 

397.136599 

0.76 

0.6076 

heursgess 

3 

30.154831 

0.10 

0.9299 

sess steam (forms heur) 

54 

4728.873047 

(no 

test) 

f ormsheur 

2 

417.487578 

0.44 

0.0526 

timesgesssf orm 

6 

186.280466 

0.B5 

0.4746 

timesgasgsheur 

3 

628.421276 

1.43 

0.2212 

timesformsheur 

2 

137.713350 

0.90 

0.4243 

sesssf orm»heur 

6 

71.502317 

0.14 

0.9910 

timesgesssf ormsheur 

L  ® 

531.349536 

1.21 

0.3170 

Since  the  same  effects  were  significant  at  comparable  levels  of 
significance  for  the  score  expressed  either  as  a  percentage  of  the 
computer  modal  score  or  as  a  raw  team  Score,  it  was  assumed  that  the 


distribution  of  difficult  trials  was  random  enough  to  wash  ofcut  any 
effects  of  differenoos  In  trial  difficulty.  Tha  remaining  variablas, 
therefore,  were  analysed  as  Simple  tsam  measures  rathtr  than  as 
percentages  of  modsl  performance. 

5.3.2  proportion  from  Oartaln-Valuad  Evants 

Behavioral  variablas  usad  to  help  axplain  parformanca  diffaranoat 
included  tha  proportion  of  a  team's  points  aarnad  from  certain-valued 
events.  Table  5.4  presents  the  ANOVA  results  for  tha  proportion  of  the 
score  from  certain-valued  events. 


TAELS  5.4 

ANOVA  Results.  Proportion  Certain 

Source 

DP 

Sum  of  Squares  Mean  Square 

Modal 

137 

9.23149951  0,04548538 

Error 

54 

0.19302872  0.00303572 

Corrected 

Total 

161 

9,39542723 

Source 

DF 

Sum  of  Squares  F  Value  Pr  >  F 

time 

l 

0,01984329 

1.08 

0,2113 

seas 

3 

0.06445043 

4,33 

0.0083 

form 

2 

0,04921023 

0,12 

0.8898 

heur 

l 

1.08747001 

9.00 

0.0287 

team(formfthaur) 

18 

3.40134981 

(no 

test) 

sossHima 

0.01925253 

1.07 

0.1833 

form»time 

0.00824509 

0.44 

0.0520 

heur»time 

1 

0.03300098 

3.97 

0.0751 

time »team(form»heur) 

18 

0.10978228 

(no 

test) 

form»sess 

0.05701037 

1.31 

0.2705 

hear «se*s 

0.04844589 

2.22 

0.0904 

sess»team(form»heur) 

54 

0.39300405 

(no 

test) 

form»heur 

2 

0.00302106 

1.73 

0.2003 

time»seea* form 

8 

0.00147469 

0.10 

0.9215 

time»sess«heur 

3 

0,02302310 

1,30 

0.2744 

tima"form«heur 

mm 

0.00108008 

0.09 

0.9149 

sess'form»heur 

■9 

0.08795494 

2.01 

0.0790 

HI 

0,02339233 

1,28 

0.2801 

There  were  no  statistically  significant  three-way  or  higher  order 
interaction  effects.  There  were  alJo  no  statistically  significant  two- 
way  interaction  affaots  for  sess*time  or  form*heur.  Using  tha  mean 
square  for  time*toam(form*heur)  as  an  error  term,  there  was  no 
statistically  significant  effeot  for  formstime  (p  *  .0529)  or  for 
heur'time  (p  ■  .0751).  Using  the  mean  square  for  sessHeamU orm»heur) 
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ai  an  arror  barm,  thara  wara  no  statistically  significant  affacta  for 
form*aasa  (p  *  .2709)  or  for  haur*sa«8  (p  *  .0004).  Sinoa  thara  wara  no 
significant  intaraotion  affactl,  ail  tha  main  afiaetl  wara  analyaad. 
Using  tha  moan  aquara  for  taam(form«haur)  aa  an  arror  tarn,  thara  was  no 
statistically  significant  affaot  for  form  (p  •  .6696),  but  thara  waa  an 
affaot  for  hour  Ip  ■  .0287).  Using  tha  maan  aquara  for 

Uma»toam(lorm»haur)  aa  an  arror  barm,  thara  waa  no  affaot  for  tima  (p  ■ 
.2113),  but,  using  tha  maan  aquara  for  sasaHaam(formnhaur)  aa  an  arror 
barm,  thara  waa  an  affaot  for  aaai  (p  •  .0063), 
s '3. ,3 

Bahavioral  variablaa  alio  inaludad  tha  proportion  of  partioular 
typaa  of  cartain-valuad  avants  to  whioh  taami  oommittad  raaouroaa. 
Tabla  9.9  praaants  tha  ANOVA  raaulta  for  tha  1-point  avants, 


TABLE  9.8  ANOVA  Results,  1-Point  Evanta  f 


Souroa 

DF 

Bum  of  Squaraa 

Maan  Sqdara 

Modal 

137 

0.73000669 

0.00932693 

Error 

64 

0.13144092 

0.00243409 

Corraotad  Total 

191 

0,30144948 

Souroa 

DF 

Bum  of  8quaraa  F 

Valua  Pr  >  F 

tima 

l 

0,01219319 

4,60 

0,0400 

aaaa 

3 

0.00613490 

0.89 

0.4B27 

form 

2 

0.01069104 

0,38 

0,6692 

haur 

1 

0.06842990 

8.17 

0,0331 

taam( f orm»haur) 

16 

0.24022166 

(no 

taat) 

sassHlma 

3 

0,00877699 

0,79 

0,6043 

f orm»tima 

0.00364987 

0.79 

0.4978 

haurotima 

0.00001302 

0.01 

0.0431 

tima» taamt form»haur) 

0.04477187 

(no 

\iast) 

f  orm»aaaa 

9 

0.01733329 

0.69 

0,4998 

haumaaaa 

3 

0,00466609 

0.68 

0.0834 

aaaantaamdormuhaur) 

94 

0,19469093 

(no 

taat) 

f orm»haur 

2 

0,07883437 

2.74 

0.0918 

9 

0.01312613 

0.60 

0,8026 

tlma»8aaa»haur 

3 

0.00426909 

0.89 

0.0289 

tlmaaformahaur 

2 

0.00829384 

1.00 

0.3977 

aaas*form»haur 

9 

0.01621892 

0.63 

0.8809 

tima»saaa»form»haur 

9 

0.00912613 

0.93 

0.7094 

Thara  wara  no  statistically  aignifioant  thraa-way  or  highar  ordar 
intaraotion  affaota.  Tharo  wara  also  no  two-way  intaraotion  affaota  for 
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seaHtime  or  form*heur.  Using  the  moan  square  for  time«team(f orm«heur) 
as  an  error  terra,  there  was  no  statistically  significant  effect  for 
formHime  (p  *  ,4075)  or  for  heur*time  (p  ■  ,0431),  Using  the  mean 
square  for  sess*tcam(f ormtheur)  as  an  srror  term,  there  wars  no  affaoti 
for  formosesi  (p  ■  ,4395)  or  lor  heur«*ess  (p  •  ,6534),  Sinoe  thera 

wars  no  significant  interaction  effects,  all  tha  main  offsets  wars 
analysed,  Using  tha  moan  square  for  taam(formithaur)  as  an  error  term, 
there  was  no  statistically  significant  affaot  for  form  (p  ■  ,0802) ,  but 
there  was  an  effect  for  hour  (p  *  ,0231),  Using  tha  mean  square  for 

timet  team(formiiheur)  as  an  error  term,  there  was  an  affaot  for  time  (p  ■ 
.0400),  but,  using  the  mean  square  for  sessRbeam(formtheur)  as  an  error 
term,  there  was  no  effeot  for  sess  (p  ■  ,4527), 

3,3,4  a.-21iv ir.-2 ‘Poln.ts.-Jar lever  Events 

Table  5.0  presents  the  ANOVA  results  for  the  4-point  events. 


TABLI  5.0 

ANOVA  Results,  4-Point  Events 

•  ► 

Source 

DP 

Bum  of  Squares 

Mean  Square 

Model 

137 

0,43331042 

0.00310200 

Evror 

54 

0,03735025 

0,00100215 

Corrected  Total 

191 

0,40005057 

8ource 

DP 

Sum  of  Squares  P  Value 

Pr  >  P 

time 

1 

0,00080208 

5,76 

0,0274 

seas 

3 

0,00104063 

0,33 

0,8115 

form 

2 

0,02020304 

1,33 

0,3104 

hour 

1 

0,05535208 

0,71 

0.0185 

team! form»heur) 

IS 

0,14850875 

(no 

test) 

sess*time 

3 

0.00429375 

1.35 

0,2587 

form»time 

2 

0.00400729 

1.31 

0.2939 

hour « time 

1 

0.00003333 

0.02 

0.8842 

timet leam(formtheur) 

IS 

0,02749375 

(no 

test) 

f ormtsess 

0 

0,00770729 

0,75 

0.0145 

heur»eese 

3 

0,00720042 

1,41 

0,3507 

*ees«team(formtheur) 

54 

0.09286135 

(no 

test) 

form»heur 

2 

0,01283239 

0.78 

0.4744 

tlmetsesatform 

0 

0,009*2188 

1.49 

0.1978 

timatsesstheur 

3 

0.016  .38," 

5,06 

0.0030 

timet form*heur 

2 

0.00700354 

2,31 

0,1377 

*ees»form»heur 

6 

0,00515531 

0,50 

0.8050 

time*eesatformtheur 

5 

0,00433339 

0,66 

0.0780 

57 


Th#r«  was  one  statistically  significant  three-way  interaction 
( timemeimheur) .  There  were  no  statistically  slgnif leant  two-way 
interaction  effect!  for  aeiiatlme  or  form»heur.  Using  the  mean  square 
for  time»team(form»heur)  al  an  error  term,  there  wee  no  itatiatloally 
lignif leant  effect  for  form*time  (p  ■  ,2030),  Doing  the  mean  square  for 
aes«»team(formNhour)  ai  an  error  term,  there  was  no  statistically 
significant  effects  for  formasess  (p  •  ,0143),  Since  form  was  the  only 
variable  not  involved  in  significant  Interaction  effeots,  only  the  main 
effects  for  form  were  analysed.  Using  the  mean  square  for 
team(formftheur)  as  an  error  term,  there  was  no  statistically  significant 
effect  for  form  (p  ■  .3104). 

3.3.9  jfcQay.j^lloJnto 

Table  0.7  presents  the  ANOVA  results  for  the  proportion  of  B-polnt 
events  completed. 


» 

TABLE  8.7  ANOVA  Results,  8-Point  Events  ‘  *  * 


Source 

DP 

Sum  of  Squares 

Moan  Square 

Model 

137 

7.83714187 

0.03730140 

Error 

34 

0.40000020 

0.00740732 

Corrected  Total 

161 

8.23714702 

Source 

DF 

8um  of  Squares  P 

Value  Pr  >  F 

time 

1 

0.23080208 

12.00 

0,0021 

seas 

0.12633098 

3,18 

0.0310 

form 

0.01243834 

0.03 

0.0752 

heur 

1 

0.14320000 

0.80 

0.4934 

teum( form»haur) 

18 

4.49080873 

(no 

test) 

seas*  time 

0.04943058 

2.04 

0.1109 

form*  time 

0.00333684 

0.14 

0,8724 

heur»time 

1 

0.07033333 

3.80 

0.0776 

time* team! form'heur) 

18 

0.30238875 

(no 

test) 

form* sees 

0.04733070 

0.60 

0.7206 

htur»soss 

0.08741637 

2.20 

0.0083 

sees* team(formkheur) 

34 

0.71443623 

(no 

test) 

formihtur 

2 

1.31186363 

2. 68 

0.0078 

time»sess*f orm 

0 

0.03201070 

0.72 

0,6340 

time*sos**heur 

3 

0.00060000 

0.43 

0.7300 

time*  form»heur 

0.00913220 

0.13 

o.een 

*eir»form*heur 

0.13070271 

1.65 

0. 1530 

timeesessMormNheur 

0.03005037 

0.00 

0.6024 

There  were  no  statistically  significant  three-way  or  higher  order 
interaction  effects.  There  were  also  no  significant  two-way  interaction 
effects  for  sess*time  or  form*heur.  Using  the  mean  square  for 
tim*«team{foi-m»liaur)  as  an  error  term,  there  was  no  significant  effect 
for  form*time  (p  *  .8724)  or  for  heur*time  (p  ■  .9776).  Using  the  moan 
square  for  sess* team( form»heur )  as  an  error  term,  there  were  no 
significant  effects  for  formesess  (p  «  .7286)  or  for  heur*sess  (p  ■ 
.0983).  Since  there  were  no  significant  interaction  effects,  all  the 
main  effects  were  analysed,  Using  the  mean  square  for  team (form* hour) 
as  an  error  term,  there  was  no  statistically  Significant  effect  for  form 
(p  «  .8898)  or  for  hour  (p  *  .4934).  Using  the  mean  square  for 
time*team(form»heur)  as  an  error  term,  there  was  an  effect  for  time  (p  ■ 
.0021),  and,  using  the  mean  square  for  sess*team(f orra»heur)  as  an  error 
term,  there  was  an  effect  for  sees  (p  *  .0310). 

5.3.6  3-P lay ar-9-Polnta -per -Player  Events  ) 

Table  5.8  presents  the  ANOVA  results  for  19-point  events.  * 


TABLE  5.8  ANOVA  Results,  15-Point  Events 


Source 

DP 

Sum  of  Squares 

Mean  Square 

Model 

137 

2.79909417 

0.02043109 

Error 

94 

0.69873750 

0.01219884 

Corrected  Total 

191 

3.49779167 

Source 

DF 

Sum  of  Squares  F 

Value  Pr  >  F 

time 

1 

0.00200208 

0.14 

0.7144 

sess 

3 

0.03395417 

0.62 

0.6023 

form 

2 

0.01073329 

0.10 

0.9081 

heur 

1 

0.04200633 

0.76 

0.3951 

team(form»heur) 

18 

0.99608750 

(no 

test) 

aess»time 

3 

0.01051042 

0.29 

0.8344 

f orm»tim* 

2 

0.09031354 

3.12 

0.0687 

hcur»tim* 

1 

0.00005208 

0.00 

0.9528 

time* team (form»heur) 

18 

0.26066250 

(no 

test) 

f orm»s*fls 

6 

0.07395521 

0.68 

0.6664 

heuMsess 

3 

0.02934583 

0.94 

0.6572 

sess* team (form»heur) 

94 

0.97881250 

(no 

test) 

form*heur 

2 

0.10501394 

0.99 

0.4057 

time*sess*form 

6 

0.03342396 

0,46 

0.8371 

time«sess*heur 

3 

0.04311879 

1.18 

0.3266 

time*  form*heur 

2 

0.03226979 

1.11 

0.3498 

seas* form"heur 

5 

0.04848229 

0.45 

0.8448 

time*s«ss*form*heur 

9 

0.00830938 

0.11 

0.9945 

There  were  no  significant  three-  or  four-way  interaction  effects. 
There  were  also  no  significant  interaction  effects  lor  seii*time  or 
form*heur.  Using  the  mean  square  for  time»team(form*heur)  as  an  error 
term,  there  was  no  significant  effect  for  form»time  (p  ■  ,0087)  or  for 
heur»time  (p  ■  .9928),  Using  the  mean  square  for  sess«team(f orm*heur) 
as  an  error  term,  there  were  no  significant  effeots  for  form»iess  (p  ■ 
.6664)  or  for  heur*seis  (p  ■  .6872).  Since  there  were  no  significant 

interaction  effects,  all  the  main  effects  were  analysed.  Using  the  mean 
square  for  toem(formNheur)  as  an  error  term,  there  was  no  significant 
effect  for  form  (p  ■  ,9081)  or  for  hour  (p  ■  .3961),  Using  the  moan 
square  for  time»team(form»heur) ,  there  was  no  effect  for  time  (p  * 
.7144),  and,  using  the  mean  square  for  sess*team(f orm*heur) ,  there  was 
no  effect  for  less  (p  ■  .0023). 

5.3.7  Peroelved  Workload 

Table  8.9  presents  the  AN0YA  results  for  workload  percSptions, 
measured  using  the  Subjective  Workload  Assessment  Technique  (SWA t) . 


TABLE  6.9  AN0VA  Results,  SWAT 


Source 

DF 

Sum  of  Squares  Mean  Square 

Model 

137 

41114.24330001  300.10396618 

Error 

84 

1722,47847222  31.89774940 

Corrected  Total 

191 

42836.72183883 

Source 

DF 

Sum  of  Squares  F  Value  Fr  >  F 

time 

11817.39421878 

98.71 

0.0001 

SOBS 

1978.07360178 

8.98 

0.0001 

form 

2012.98108079 

1.06 

0.3682 

heur 

1 

79.07472367 

0.08 

0.7768 

team(form»heur) 

18 

17148.11010417 

(no 

test) 

sees* time 

184.09848032 

1.02 

0.1962 

form»time 

319.81362196 

1.32 

0.2924 

heur»timt 

1 

71.70370370 

0.60 

0.4490 

time*  team (form»heur) 

18 

2199.03434028 

(no 

test) 

form»sess 

198.47398761 

0.48 

0.8416 

heumsess 

229.60792101 

1.02 

0.3892 

sees* team (form* heur) 

64 

3900.00142361 

(no 

test) 

form*heur 

2 

236.41929320 

0.12 

0.8840 

tJme*iesi»form 

6 

88.30088903 

0.30 

0.9317 

Ume»sesi*heur 

3 

13.68769076 

0.14 

0.9339 

time*  form»heur 

2 

72.46984447 

0.30 

0.7428 

sees* form»heur 

mm 

648.26482908 

1.24 

0.2988 

time* sees* form* heur 

■9 

68.31087138 

0.34 

0.0119 

There  were  no  statistically  significant  three-way  or  higher  order 
interaction  effects.  Thar#  wara  alio  no  aignificant  two-way  interaction 
effect!  for  soas»time  or  form»heur,  Uaing  the  mean  aquara  for 
time* team(form»heur)  ai  an  error  term,  there  wai  no  atatiitically 
aignificant  affect  for  formxtime  (p  •  .2024)  or  for  heur*tlme  (p  ■ 
,4400).  Uaing  the  mean  aquara  for  aeaaxteam(form»heur)  aa  an  error 
term,  there  ware  no  aignificant  effect!  for  formxaeaa  (p  ■  .8410)  or  for 
heur«a«aa  (p  ■  .3802).  Since  there  were  no  aignificant  interaction 
effect!,  all  the  main  effacta  were  analysed.  Uaing  the  mean  aquara  for 
team(form»heur)  aa  an  error  term,  there  waa  no  aignifioant  effect  for 
form  (p  *  .3682)  or  for  hour  (p  ■  .7769).  Uaing  the  mean  aquara  for 
time«team(form«heur)  aa  an  error  term,  there  waa  an  effeot  for  time  (p  ■ 
.0001),  and,  uaing  the  mean  aquara  for  aeiaxteam(formxhour)  aa  an  error 
term,  there  waa  an  effeot  for  aeaa  (p  ■  .0001). 

9.4  ANOVA  Model  for  Information  Seeking  * 

The  model  uaed  for  the  atatiatioal  analyaia  of  the  proportion  of 
uncertain  event!  about  which  team!  sought  additional  information  waa 

Yukimn  •  u.  ♦  time*  ♦  aaesj  +  type*  ♦  heun  +  tearnm<i>  ♦ 
(time«aeaa) i j  ♦  (time«type) im  ♦  (tlmexheur) tx  *  (timexteam) im< » >  ♦ 
,  (aeaaxtype) ♦  (aeaaxheur) t i  ♦  (aeaa » team) t m< i >  ♦  (type*heur)m  + 
(typexteam)km< i >  +  (time»aeaa»type) uk  ♦  (timexaeuaxheur) ui  ♦ 

(time*a«aaxteam) i4m< i >  ♦  (time»type*heur) ,ki  + 

(timextypexteam)  tkm<  t  >  +  (aeaaxtypexheur)  jm  ♦ 

(aeaa»typexteam)  jum« i >  +  (timexaeaaxtypexheur)  t jMt  +  emitkim 

where 

*Y‘  repreaenta  the  proportion  of  uncertain  event!  about  which 
additional  information  la  sought 

'time'  la  the  time  atreaa  variable  (high  or  moderate  time  stress) 
'aeaa'  ia  the  practice  variable  (session  1,  2,  3,  or  4) 

'type'  ia  the  specific  type  of  uncertain  event  involved  (H21,  H0, 
LB4 ,  or  L36) 

‘hour*  is  the  suggested  decision  strategy  variable  (with  or  without 
heuristics) 

'team*  is  a  variable  that  takes  into  account  any  differences  among 
the  12  teams  aasigned  to  each  heuristics  treatment, 
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The  information  presentation  form  variable  was  not  included  in  the 
model  because  the  symbols  for  uncertain  events  (black  rectangles)  were 
identical  for  all  treatment  groups .  (To  confirm  that  the  information 
presentation  form  variable  was  not  rsquirad  in  ths  model,  a  model 
including  the  form  variable  was  anaiysad,  There  were  no  statistically 
significant  interaction  affeots  involving  information  presantation  form, 
nor  was  than  a  main  affeot  lor  form  (p  ■  .9468).) 


TABLE  9.10  ANOVA  Modal  for  Information  Staking 


Source 

DF 

Expected  Mean  Square 

time 

1 

384Iai*  ♦  lflc..'  ♦  C 

seas 

3 

192IbjV3  ♦  8c*.'  ♦  c' 

type 

3 

192Ioh*/3  +  Be.'  ♦  o* 

heur 

1 

aa4Xda*  ♦  32c.  •  +  C 

team(heur) 

22 

32c.'  ♦  c'  ) 

aeai* time 

3 

BOEZ(ab)  u*/3  +  4c.*.'  +  o'"  ' 

type»time 

3 

96££(ao)n,V3  +  4c...'  +  c'  1 

heur«time 

1 

192SZ(ad) it*  ♦  10s*#'  ♦  o* 

time*team(h#ur) 

22 

10c..*  ♦  c' 

type»aeaa 

9 

48XX (bo) ik*/9  ♦  Boa..'  +  c“ 

heur*eoaa 

3 

POIKbdJji'/S  ♦  8c*.-  ♦  o' 

sesi> tsam(heur) 

06 

89m1  ♦  »' 

type»heur 

3 

96II(od)i.»V3  +  So,.'  ♦  o' 

type* teamlheur ) 

Bo..'  ♦  o' 

W./i' 

24£II(aba)  uw'/P  ♦  0.M*1  ♦  o* 

time*eeaa*heur 

3 

48£££(abd)  ui'/3  ♦  4c**.'  ♦  c' 

time«sess»team(heur) 

66 

4c**.'  ♦  c' 

ti mao  type* heur 

3 

46Z££(acd) ♦  4c.*.'  ♦  c' 

time* type«team(heur) 

66 

4c.**'  +  c' 

aeas»typ#»heur 

9 

24£££(bcd)  jm'/P  ♦  2c*«.*  +  C* 

sosa* type* team (heur) 

198 

2c*..'  ♦  c' 

time*sess*type*heur 

9 

12££££(abod)utai'/9  ♦  c.*..'  *  c* 

time* seas ‘type* teamlheur) 

198 

c.**.'  ♦  c' 

error 

0 

c* 
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Table  9.10  ahewa  the  model ' a  degrees  ol  freedom  and  expected  mean 
squares.  A  four-way  interaction  tarn,  (time*aeas* type* team)  i jhmt  i > ,  waa 
pooled  into  tha  error  tarn  ainoe  no  degrees  of  fraedon  remained  lor  the 
error  tern.  Aa  a  reault,  no  teata  of  team  affecta  or  of  tha  aflacta  of 
interactiona  between  tha  team  variable  and  other  variablaa  were  provided 
by  the  model,  but  team  affeota  were  not  a  focua  of  thia  atudy. 

Teata  of  the  aaaumptiona  on  which  tha  ANOVA  model  la  baaed 
demonstrated  that  tha  modal  id  applicable  to  tha  reaulta  of  thia  atudy, 
Specif icaily,  within  oommoniy  aooepted  limita,  all  obeervatione  have  tha 
same  variance  and  the  error  terma  are  normally  diatributad. 

5.9  8 

Table  9.11  ia  the  AHOVA  table  for  tha  proportion  of  the  uncertain 
events  about  which  a  team  sought  additional  information, 


TABLE  9.11 

Source 

Model 

Error 

Corrected  Total 
Source 


ANOVA  Reaulta,  Information  Seeking  V 

OF  Sum  of  8quares  Mean'Squkre 
990  31.03940740  0.09912741 

106  2.14011458  0,01089411 

797  34,08991108 

DF  Sum  of  Squares  F  Value  Pr  >  F 


time 

1 

1.49299208 

73.82 

0.0001 

seas 

3 

0.27010159 

4.00 

0.0112 

type 

3 

0,08044010 

2.55 

0.0931 

heur 

1 

7.14040210 

0.81 

0.0048 

team(heur) 

22 

19.03391070 

(no 

teat) 

aessutime 

3 

0.03177003 

1.43 

0.2429 

type»time 

3 

0.03002904 

0.02 

0.1343 

heur* time 

1 

0.02920833 

1.28 

0.2900 

timenteamtheur) 

22 

0.43289458 

(no 

teat) 

type«sess 

0 

0.11373908 

1.09 

0.4010 

heur»aeaa 

3 

0.10039199 

1.44 

0.2400 

seaa«team(heur) 

99 

1.93970937 

(no 

teat) 

type»heur 

3 

0.01720052 

0.40 

0.9001 

type»leam(heur) 

99 

0.77148438 

A  1111 OOI 1 

(no 

1  1  1 

teat) 

A  *t  1  AQ 

time»aeaa*heur 

V 

3 

vi  11 11V4 f  1 

0.00597708 

1  e  1  r 

0.39 

0.8574 

tlme«aeaa»team(heur) 

99 

0.48042708 

(no 

test) 

t ime»type*heur 

3 

0.00742188 

0.23 

0.8793 

time* type* team (heur) 

99 

0.71509208 

(no 

teat) 

aeaa»type*heur 

0 

0.00858073 

0.01 

0.5197 

aeaa«type«team(heur) 

108 

2.38080720 

(no 

teat) 

t ime»a*aa»type»h*ur 

0 

0.09100271 

0.93 

0.7603 

There  were  no  statistically  significant  three-way  or  higher  order 
interaction  effects.  There  were  alio  no  significant  two-way  interaction 
effects  for  scssxtime,  typextime,  sessxtype,  or  typexheur.  Using  the 
mean  square  for  timexteam(heur)  as  an  error  term,  there  was  no 
significant  effect  for  heurxtime  (p  *  .2600).  Using  the  mean  square  for 
sessxteam(heur)  as  an  error  term,  there  was  no  significant  effect  for 
heur»sess  (p  ■  .2400).  Since  there  were  no  significant  interaction 

effects,  all  the  main  effects  were  analyzed.  Using  the  mean  square  for 
team(heur)  as  an  error  term,  there  was  an  effeot  for  heur  (p  ■  .0040), 
Using  the  mean  square  for  timexteam(heur)  as  an  error  term,  there  was 
also  an  effect  for  time  (p  •  .0001),  and,  using  the  mean  square  for 
less* team(heur)  as  an  error  term,  there  was  an  effeot  for  sees  (p  ■ 
.0112).  Using  the  mean  square  for  typexteara(heur)  as  an  error  term, 
there  was  no  statistically  significant  effeot  for  type  (p  ■  .0631). 

5.6  ANOVA  Model  for  Resource  Commitments  to  Uncertain  Events  ? 

The  model  used  for  the  statistical  analysis  of  the  proportion  of 
uncertain  events  to  which  teams  committed  resources  subsequent  to 
seeking  additional  information  was 

Ytjhim  ■  u.  ♦  timet  ♦  type*  ♦  hour*  ♦  teamton  ♦  (time«type) u 
♦  (time«heur) tw  +  (timexteam) t »<*>  +  (typexheur) j*  ♦ 

(typexteam)  jimt  ♦  (time* typexheur)  u*  +  emtum> 

where 

* Y*  represents  the  proportion  of  uncertain  events  to  which 
resources  are  committed  subsequent  to  seeking  additional  information. 
It  is  measured  in  terms  of  a  ratio  of  the  number  of  uncertain  events 
completed  to  the  number  of  uncertain  events  about  which  additional 
information  was  sought  (not  as  a  ratio  of  uncertain  events  completed  to 
total  uncertain  events  presented) . 

'time'  is  the  time  stress  variable  (high  or  moderate  time  stress) 
‘type*  represents  the  specif ic  type  of  uncertain  event  involved 
(H2 1 ,  H0 ,  L84 ,  or  L36) 

'heur*  is  the  suggested  decision  strategy  variable  (with  or  without 
heuristics) 

‘team'  is  a  variable  that  takes  into  account  any  differences  among 
the  12  teams  assigned  to  each  heuristics  treatment. 
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The  experimental  modal  used  to  analyst  resource  commitments  to 
uncartaln  avanta  doaa  not  lnoludt  tha  practice  varlabla.  A  vary  few 
taami  chose  to  seek  additional  information  about  ao  itw  unoartain  avanta 
that,  for  a  given  axperimantal  aaaaion,  it  was  poaaibla  that  a  taam  had 
no  unoartain  avanta  of  aoma  particular  typa  aval labia  to  oomplata. 
Sinca  proportions  with  denominators  of  zero  oould  not  ba  analysed,  aoma 
altarnata  analysis  mechanism  was  required.  Data  involving  denominators 
of  eero  oould  ba  ignored,  creating  unequal  call  aisaa.  Tha  SAS  General 
Linear  Modala  procedure  provided  for  analyaia  involving  unequal  cell 
sizes,  but  oomputer  memory  limitations  praoludad  analyaia  with  a 
oomplata  modal,  An  analyaia  of  a  limited  modal  (only  main  af facta  and 
two-way  lntaraotiona  involving  the  time  pressure,  praotloa,  event  type, 
heuristics,  and  taam  variables)  revealed  that  there  waa  no  main  effect 
for  tha  practice  variable  nor  wore  there  any  two-way  interaction  aftaota 
involving  the  praotloa  variable,  Therefore,  tha  praotloa  varilbla  was 
dropped  from  tha  modal,  and  data  for  each  taam  waa  averaged  ao/oss  tha 
four  sessions,  Further,  since  every  team  sought  additional  information 
about  at  least  one  unoartain  event  of  each  type  sometime  during  the 
experiment,  no  data  Involving  denominators  of  zero  remained.  Therefore, 
a  modal  ignoring  the  session  variable  oould  be  analysed  using  the  BAS 
ANOVA  procedure, 

The  information  presentation  form  variable  waa  not  lnoluded  in  the 
model  because  the  aymbols  for  unoartain  events  (black  rectangles)  were 
Identical  for  all  treatment  groups.  (To  confirm  that  the  information 
presentation  form  variable  was  not  required  in  the  model,  a  model 
including  the  form  variable  was  analysed.  There  were  no  statistically 
significant  interaction  effects  involving  information  presentation  form, 
nor  was  there  a  main  effect  for  form  (p  ■  .6338).) 

Table  6.12  provides  the  degrees  of  freedom  and  expeated  mean 
squares  for  the  model,  A  three-way  interaction  term, 
(time'typeKteam)  01  <i«> ,  was  pooled  into  the  error  term  slnoe  no  degrees 
of  freedom  remained  for  the  error  term.  As  a  result,  no  tests  of  team 
effects  or  of  the  effeots  of  interactions  between  the  team  variable  and 
other  variables  were  provided  by  the  model,  but  team  effects  were  not  a 
focus  of  this  study. 
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TABLE  5.12  ANOVA  Modal  lor  Raaourca  Commitments  to  Uncertain  Events 


Source 

DF 

Expected  Mean  Square 

time 

1 

BSXai*  ♦  4c*4#  ♦  o* 

type 

3 

46Zb*a/3  ♦  2cna  ♦  9* 

heur 

1 

86Xc««a  4  8oa“  ♦  o* 

team(heur) 

22 

8C*a  ♦  ca 

type»time 

,  3 

24Z£(ab)  i  j*/3  4  c.m*  4  oa 

heur* time 

1 

48XX(ao)ika  4  4c»4a  4  oa 

time* team(heur) 

22 

4e*aa  4  ca 

type*heur 

3 

24ZZ(bo) jua/3  4  2cw4a  4  oa 

type* team(heur) 

68 

2ffw«a  4  oa 

time* type»heur 

3 

12XZZ(abc)  uka/3  4  9.Ma  4  oa 

time* type* team(heur) 

66 

C*Ha  ♦  Ca 

error 

0 

Ca 

> 

Taste  of  the  assumptions  on  which  the  ANOVA  modal  1'S  based 
demonstrated  that  tha  model  Is  applicable,  with  reservations,  to  the 
results  of  this  study,  Speolfioally,  the  variance  for  teams  with 
heuristics  was  smaller  than  for  teams  without  heuristics.  The  error 
terms  were  approximately  normally  distributed.  No  standard 
transformation  reduced  the  difference  in  variability.  Since  the  test 
was  at  the  commonly  accepted  limit  and  no  better  model  existed,  the 
model  was  accepted  as  applicable  for  the  results  of  the  study.) 

5 • 7  Statistical  Analysis  UslnS  ANOVA  Model  for  Resource  Commitments  to 
Urcejrtain  Events 

The  variable  used  to  measure  resource  commitments  to  unoertain 
events  was  the  proportion  of  unoertain  events  of  a  particular  type  to 
which  resources  were  committed  subsequent  to  seeking  information.  Table 
5.13  is  the  ANOVA  table  for  the  resource  commitment  variable. 

The  three-way  interaction  (time»type»heur)  was  not  statistically 
significant.  There  was  also  no  statistically  significant  two-way 
interaction  effects  for  type*time.  Using  the  mean  square  for 
timciteam(heur)  as  an  error  term,  there  was  no  statistically  significant 
effect  for  heur*time  (p  ■  .1368).  Using  the  mean  square  for 
type»team(heur)  as  an  error  term,  there  was  an  effeot  for  type*heur  (p  * 
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.0001).  Since  only  time  was  not  involved  in  any  significant  interaction 
effects,  only  the  main  effects  for  time  were  analysed.  Using  the  mean 
square  for  time»team(heur)  as  an  error  term,  there  was  no  statistically 
significant  effect  for  time  (p  ■  .9874). 


TABLE  8.13  ANOVA 

Results,  Resource  Commitments  to 

Uncertain  Events 

Source 

DF 

Sum  of  Squares 

Mean  Square 

Model 

125 

34.07905844 

0,27263727 

Error 

60 

0,22582388 

0.00342157 

Corrected  Total 

191 

34.30648231 

Source 

DF 

Sum  of  8quares  F 

Value  Pr  >  F 

time 

1 

0.00000170 

0.00 

0.9874 

type 

3 

22.91712950 

110.98 

0.0001 

heur 

1 

1.10962269 

10.05 

0.0036 

team(heur) 

22 

2.29129487 

(no 

test) 

typeatime 

3 

0.01171740 

1.14 

0.3389 

heur atime 

1 

0.01579871 

2.30 

0.1380 

timeateam(heur) 

22 

0.14708877 

(no 

test)* 

typeaheur 

3 

3.042B1 147 

14.74 

0.0001 

typeateam(heur) 

66 

4,84287261 

(no 

test) 

timeatypeaheur 

3 

0.00134371 

0.13 

0.9414 

*'e  Findings,  &1H1&  to  tfte  flaaMilil 

The  following  substations  relate  the  findings  outlined  above  to  the 
hypotheses  presented  in  Chapter  3. 

5.0.1  Effects  of  Information  Presentation  Form 

This  research  compares  the  impact  of  three  coding  schemes:  two 
single-featured  schemes  (color  and  alphabetic)  and  a  conjunctive  scheme 
(combination  of  color  and  alphabetic).  The  following  two  subsections 
repeat  the  hypotheses  and  present  findings  concerning  the  effects  of 
information  presentation  form  on  decision  quality  and  peroeived 
workload. 


B.e.  1.1  i9rR..inl 

Ilia .  The  quality  of  team  decisions  in  the  TRAP  will  be  higher 
for  teams  confronted  with  single-featured  (either  alphabetic 
or  color-coded)  event  symbols  than  for  teams  confronted  with 
events  represented  by  conjunctions  of  features.  However, 
there  will  be  no  difference  in  decision  quality  between  teams 
using  an  alphabetic  coding  scheme  and  teams  using  a  color 
coding  soheme. 
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The  primary  measure  of  decision  quality  in  performing  the  TRAP  was 
the  ratio  of  a  team's  soore  on  a  trial  to  the  computer  model's  score  on 
the  same  trial.  The  ratio  of  a  team's  points  earned  from  certain-valued 
events  to  the  team's  total  points  provided  a  behavioral  measure  to  help 
explain  overall  performance  differences;  high  quality  deolsion  making  is 
indicated  by  approaching  the  ratio  achieved  by  the  computer  model,  Data 
on  the  proportions  of  various  types  of  certain-valued  events  completed 
was  also  analysed;  high  quality  decision  making  is  indicated  by  low 
proportions  of  low-value  events  completed  and  high  proportions  of  high- 
value  events  completed. 

The  results  of  this  study  suggest  that  information  presentation 
form  does  not  affect  overall  deolsion  quality  on  the  TRAP.  Using  the 
ratio  of  team  soore  to  model  soore  as  a  measure,  there  were  no 
significant  differences  among  the  form  treatments  (p  •  .0330).  Teams 
using  the  color  coding  scheme  earned  74.4  percent  of  the  points  earned 
by  the  model  on  the  same  trials,  teams  using  the  alphabetic  Schd'me  73.0 
percent,  and  teams  using  ths  conjunctive  scheme  72.7  percent.  It  should 
be  noted  that  the  tests  were  powerful  enough  to  detect  effeots  of  other 
variables.  The  power  of  the  P  test  to  detect  differences  among 
information  presentation  form  treatments  of  4  psreent  of  ths  model  soore 
(comparable  to  other  detected  differences)  given  the  variability 
experienced  in  this  study  exoeedsd  90  percent. 

The  ratio  of  team  score  to  model  soore  is  an  overall  measure  of 
team  performance,  but  the  same  level  of  performance  could  be  achieved  in 
a  variety  of  ways.  Othsr  measures  that  dtsorlbt  how  a  team  behaved  with 
respect  to  speolfic  features  of  the  TRAP  help  to  explain  differences  in 
performance.  Specifically,  two  additional  measures  were  used  in  this 
study.  The  relative  emphasis  that  a  team  plaoad  on  certain  and 
uncertain  events  was  captursd  in  the  ratio  of  a  team's  points  earned 
from  certain-valued  events  to  ths  tsam's  total  points.  At  a  finer  level 
of  detail,  a  second  behavioral  measure  examined  the  proportion  of  events 
of  each  type  completed  by  a  team. 

Using  the  ratio  of  a  team's  points  earned  from  oertain-valuad 
events  to  the  team's  total  points  as  a  measure,  there  were  still  no 
signtf leant  differences  among  ths  information  presentation  form 
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treatments  (p  *  ,8080),  Teams  uiing  the  ooior  coding  Scheme  earned  50.1 
percent  of  thair  pointa  from  aertein-valued  avanta,  teams  uaing  tha 
alphabetic  achama  52.0  parcant,  and  taama  uaing  tha  conjunctiva  achama 

48.2  parcant,  By  compariaon,  tha  modal  aarnad  45,7  parcant  of  ita 
pointa  from  certain-valued  avanta. 

Similarly,  information  praaantation  form  had  no  affaata  on  tha 
proportion!  of  apaolfic  certain-valued  avanta  oompleted.  Completion 
rates  for  two  low-value  avanta  (1-playar-l-point-par-playar  and  2- 
playar-2-pointa-par-playar)  and  two  high-value  avanta  (3-player-B- 
pointa-per-player  and  2-playar-4-pointa-par-playar)  ware  analysed.  A 
third  type  of  low-value  event,  3-player-l-point-per-player ,  oould  not  be 
atatiatically  analysed  since  only  one  such  event  waa  oompleted  by  any 
team  (a  team  uaing  the  conjunctive  coding  scheme),  Thera  ware  no 
significant  diffaranaaa  among  tha  form  treatments  for  completion  rates 
of  any  of  tha  analysed  event  typen  * 

*  y 

1- playar-l-point-par-playar  p  *  .8082 

2- playar-2-pointa-par-playar  p  ■  .3164 

3- player-B-pointa-per-player  p  •  .8081 

2-player-4-pointa-per-player  p  ■  .8752 

Hypothesis  Hla,  than,  waa  partially  supported  by  tha  results  of 
this  atudy,  Tha  findings  supported  tha  expected  lack  of  difference  in 
decision  quality  between  teams  uaing  different  single-featured  coding 
schemes.  However,  tha  results  did  not  support  tha  expected  difference 
in  decision  quality  between  taama  uaing  aingla-f eaturad  schemes  and 
those  using  a  conjunctive  soheme. 

3.8. 1.2  Information  Presentation  Form  and  Perceived  Workload 

Hlb.  Tha  peroalvad  workload  in  the  TRAP  will  be  lower  for 
teams  confronted  with  single-featured  (either  alphabetic  or 
' color-coded)  event  symbols  than  for  teams  confronted  with 
events  represented  by  conjunctions  of  features .  However, 
there  will  be  no  difference  in  peroeived  workload  between 
teams  using  an  alphabetic  ooding  scheme  and  teams  using  a 
color  coding  scheme, 

The  single  measure  of  perceived  workload  in  performing  the  TRAP  was 
the  average  of  team  member  Subjective  Workload  Assessment  Technique 
(SWAT)  ratings.  SWAT  ratings  may  range  on  a  soale  from  0  to  100  (see 
Appendix  A) ,  The  results  of  this  study  suggest  that  information 


presentation  form  does  not  affect  perceptions  of  workload  associated 
with  performing  the  TBAP,  There  were  no  significant  differences  in  SWAT 
ratings  among  the  information  presentation  form  treatments  (p  *  .3082), 
Hypothesis  Hlb,  then,  was  partially  supported  by  the  results  of 
this  study.  The  findings  supported  the  expected  lack  of  difference  in 
perceived  workload  between  teams  using  different  single-featured  coding 
schemes.  However,  they  did  not  support  the  expected  difference  in 
perceived  workload  between  teams  using  single-featured  sohemea  and  those 
using  a  conjunctive  aoheme. 

5.8.2  Effects  of  Suggested  decision  Strategy 

This  research  assesses  the  impact  of  provision  and  discussion  of 
decision  heuristios  by  comparing  performance  with  the  heuristics  to 
performance  without  the  heuristics.  The  following  two  subseotions 
repeat  the  hypotheses  and  present  findings  concerning  the  effeots  of  the 
suggested  deaision  strategy  on  deoision  quality  and  peroeived  workload. 

5 . 8 . 2 . 1  Suggested  Decision  Strategy  and  Deoision  Quality  *  * 

Hypotheses  and  findings  concerning  the  effeots  of  the  suggested 
deoision  strategy  on  deoision  quality  are  presented  in  two  parts, 
Effects  on  overall  decision  quality  will  be  addressed  first  followed  by 
an  assessment  of  the  particular  effeots  on  the  quality  of  decisions  made 
under  uncertainty. 

b. a. 2. i.i  giiifllua  amiU  Bialilaa. Aiall to- 

H 2 a ( 1 ) ,  The  quality  of  team  deoision  in  the  TRAP  will  be 
higher  with  provision  and  discussion  of  decision  heuristios 
than  without  provision  and  discussion  of  deoision  heuristics. 

In  this  study,  overall  decision  quality  was  affected  by  the 

presence  or  absence  of  a  suggested  decision  strategy.  Ratios  of  beam 

scores  to  model  scores  were  significantly  higher  with  heuristics  (75. B 

percent  of  model)  than  without  heuristios  (71.2  percent  of  model)  (p  • 

.0082) . 


Using  the  ratio  of  a  team's  points  earned  from  certain-valued 
events  to  the  team's  total  points  as  a  behavioral  measure,  teams  with 
heuristics  earned  a  significantly  lower  proportion  of  their  points  from 
certain-valued  events  (42.6  percent)  than  did  teams  without  heuristic! 
(57.8  percent)  (p  ■  .029),  For  comparison,  the  model  earned  45.7 


percent  o(  its  points  from  certain-valued  events.  Teams  with  heuristics 
behaved  more  like  the  model  than  did  teams  without  heuristics. 

The  heuristics  treatment  also  affected  the  proportions  of  some 
specific  certain-valued  events  completed;  teams  with  heuristics 
completed  fewer  low-value  events  (as  recommended  in  the  heuristics)  than 
did  teams  without  heuristics.  Specif ically,  teams  with  heuristics 

completed  3.5  percent  of  the  i-player-l-polnt-per-player  events  compared 
to  the  7.7  percent  completed  by  teams  without  heuristios  (p  •  .0231). 

Hypothesis  H2a ( 1 ) ,  then,  was  supported  by  the  results  of  this 
study.  The  findings  supported  the  expected  difference  in  deoision 
quality  between  teams  with  and  those  without  the  suggested  decision 
strategy. 

0*8.2. 1 .2  Effects,  on  the  Quality  of  Decisions  Under  Uncertainty. 

I 

H2a(2) .  Without  the  provision  ' end  discussion  of  deoision 
heuristics i  teams  will  seek  additional  Information  about  and 
commit  resources  to  uncertain  (.vents  at  o  rate  different  from 
that  anticipated  according  to1  expected  utility  theory.'  With 
the  provision  and  discussion  of  decision  heuristics,  teams 
will  seek  additional  information  about  and  commit  resources  to 
uncertain  events  according  id  sweated  utility  theory. 

Difterenoes  in  how  teams '  dealt  with  uboei  tainty  were  examined  in 

terma  of  the  proportion  of  uncertain  events  about  which  teams  sought 

additional  information  and  the  proportion  of  uncertain  events  tn  which 

reeources  were  committed  subsequent  to  seeking  additional  information. 

5.0.2, 1.2.1  Seeking  Information  About  Uncertain  Events.  Thv  results  of 

this  study  suggsst  that  the  presence  or  absence  of  heuristics  Coes 

a  I  feat  tho  proportion  of  uncertain  events  about  which  teams  seek 

additional  information  (p  ■  .0049).  Teams  with  heuristics  seek 

information  about  significantly  more  uncertain  events  (57.9  percent) 

than  do  teams  without  heuristics  (38.0  psreent) .  For  comparison,  the 

computer  model,  which  uses  expected  value  as  ths  basis  for  its  decisions 

about  uncertain  events,  sought  information  about  05.7  percent  of  the 

uncertain  event!. 

The  portion  of  hypothesis  H2a(2)  related  to  seeking  information 
about  uncertain  events,  then,  was  supported  by  the  results  of  this 
study,  Teams  with  the  euggested  decision  strategy  sought  additional 
information  about  uncertain  events  at  a  rate  more  consistent  with 
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expected  utility  theory  than  did  teams  without  the  suggested  decision 
strategy, 

5. G. 2. 1.2, a  Committing  Resources  to  Uncertain  Events,  The  results  of 
this  study  suggest  that  the  presence  or  absence  of  heuristics  does 
a  ( loot  the  proportion  of  uncertain  events  to  whioh  teams  commit 
resources  subsequent  to  seeking  additional  information.  Specifically , 
the  interaction  between  the  heuristics  treatment  and  the  types  ol 
uncertain  events  was  significant  (p  •  ,0001).  Pour  types  of  uncertain 
events  ware  defined  by  the  combination  of  an  event's  probability  of 
payoff  1’H‘igh  or  ’L'ow)  and  its  possible  point  value  (0,  21,  30,  or 
84).  The  possible  combinations  were  H9,  )Ul,  LOO,  and  L84 .  Table  8.14 
provides  the  proportions  of  aaoh  type  of  uncertain  event  completed  by 
teams  with  and  teams  without  heuristics.  Proportions  completed  by  the 
computer  model  are  alS'  included  for. comparison. 

i  •  ; 

V 

TABLE  0.14  Lasouros  Commitments  to  Uncertain  trents  * 

1121  H9  ,  '  L84  139 


With 

Heuristics 

93.  IX 

0.  IX 

91. ex 

0.BX 

Without 

■: 

Heuristics 

99.  OX 

48. 9X 

T3.0X 

31.  BX 

Model 

100. OX 

4.  or. 

100. OX 

4.2X 

Figure  6.1  graphically  illustrates  the  Interaction  when  date  points 
are  computed  as  the  difference  between  the  proportion  of  uncertain 
events  completed  by  teams  and  thole  completed  by  the  model,  (Tho  aero 
line  on  the  figure  reflects  performance  by  the  model.)  The  figure  shows 
that  teams  with  heuristics  committed  resources  to  uncertain  events  at 
rates  very  similar  to  the  model.  Teams  without  heuristics  matched  the 
model  for  events  with  a  high  probability  of  a  large  payoff  (H21)  but 
deviated  from  the  model  on  other  events.  Specifically,  teams  without 
heuristics  overcommitted  to  events  with  low  expected  utility  (H9  and 
L30)  and  undtrcommltted  to  averts  with  high  expected  utility  but  a  low 
probability  of  payoff  (L84) 


Type*heur  Interaction  Plot 

Dlffereneae  tram  Mortal 

Poroont  Oomplolod 


Figure  6,1  Resource  Commitments 

The  portion  of  hypotheaia  H2a(2)  related  to  reaource  commitment  to 

uncertain  avanta,  than,  waa  aupportad  by  tha  raaulta  of  thia  atudy. 

Tearna  with  tha  auggaatad  daoiaion  atratagy  oommittad  raaouroaa  to 

uncertain  avanta  at  a  rata  more  oonaiatant  with  expeotod  utility  theory 

than  did  taama  without  tha  auggaatad  daoiaion  atratagy. 

S.8.2.2  Sukrteated  Daoiaion  Strategy  and  Perceived  Workload 

H2b.  Tha  peroeived  workload  in  tha  TRAP  will  be  lower  with 
provialon  and  dlacuaalon  of  daoiaion  haurlatioa  than  without 
proviaion  and  diaouaaion  of  daoiaion  haurlatioa. 

In  thia  atudy,  peroeivad  workload,  aa  meaaurad  by  SWAT  ratinga,  waa 

not  affactad  by  tha  preaence  or  abaenoe  of  a  auggaatad  daoiaion  atratagy 

(p  *  .7765).  Taama  with  haurlatioa  rated  tha  TRAP  at  30.5  on  the  100- 

point  workload  aoala;  taama  without  haurlatioa  rated  tha  TRAP  at  38.2. 

Hypothaaia  H2b,  than,  waa  not  aupportad  by  tha  raaulta  of  thia  atudy. 

5.8.3  Effaota  of  Tima  Straaa 

Thia  raaaaroh  oomparaa  tha  impact  of  two  lavala  of  time  atrasa. 
Tha  following  two  aubaaotiona  repeat  tha  hypothaaaa  and  praaent  flndinga 
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concerning  the  effects  of  time  stress  on  decision  quality  and  perceived 
workload. 

5.0.3. 1  Time  Stress  and  Deoislon  Quality. 

H3a.  The  quality  of  team  deolsions  in  the  TRAP  will  be  higher 

under  moderate  time  pressure  than  under  high  time  pressure. 

The  results  of  this  study  suggest  that  time  stress  does  affect 
overall  decision  quality  on  the  TRAP.  Using  the  ratio  of  team  soore  to 
model  score  as  a  measure,  teams  performed  significantly  better  under 
moderate  time  pressure  (79.9  peroent  of  the  model  score)  than  under  high 
time  pressure  (07.2  percent  of  the  model  score)  (p  ■  .0001). 

Time  pressure  also  had  a  main  effect  on  the  proportions  of  1- 
player-  1-point  and  2-player-4-pointS-per-player  events  completed.  In 
each  case,  teams  completed  more  events  under  moderate  time  pressure  than 
under  high  time  pressure.  Table  8. IB  shows  the  proportions  of  the  two 
types  of  events  completed  under  high  and  moderate  time  pressureyand  the 
significance  of  the  differences.  Proportions  completed  by  the  model  are 
also  shown  for  comparison.  Note  that,  although  teams  came  closer  to 
behaving  like  the  model  under  moderate  time  pressure  than  under  high 
time  pressure,  it  was  not  necessarily  ‘good*  to  complete  more  low-valued 
events. 


TABLE  9.19.  Events  Completed  versus  Time  Pressure 

1  Player  2  Players 

1  Point  4  Points  each 


Model 

.073 

.569 

High 

Pressure 

.048 

.382 

Moderate 

Pressure 

.004 

.438 

Significance 

p  ■  .040 

p  ■  .002 

Time  stress  also  affeoted  information  seeking  activity.  Teams 
sought  additional  information  about  Significantly  more  uncertain  events 
under  moderate  time  pressure  (82.0  peroent)  than  under  high  time 
pressure  (43.9  percent)  (p  ■  .0001).  For  comparison,  the  computer  model 
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sought  information  about  65.7  percent  of  the  uncertain  events.  Teams 
behaved  more  like  the  model  under  moderate  time  pressure  than  under  high 
time  pressure. 

Hypothesis  H3a,  then,  was  supported  by  the  results  of  this  study. 
Except  for  the  number  of  low-valued  events  completed,  the  findings 
supported  the  expected  degradation  of  decision  quality  under  high  time 
pressure. 

3. 8. 3. 2  Time  Stress  and  Perceived  Workload 


H3b.  The  perceived  workload  in  the  TRAP  will  be  lower  under 
moderate  time  pressure  than  under  high  time  pressure. 

Using  SWAT  ratings  as  a  measure,  team  perceptions  of  workload  in 

trials  under  moderate  time  pressure  were  significantly  lower  (31.0)  than 

in  trials  under  high  time  pressure  (46.7)  (p  *  .0001).  Hypothesis  H3b, 

then,  was  supported  by  the  results  of  this  study. 

5.0.4  Effects  of  Practice  * 

This  research  assesses  the  impact  of  praatice  on  TRAP, performance. 

The  following  two  subseotions  repeat  the  hypotheses  and  present  findings 

concerning  the  effects  of  practice  on  decision  quality  and  perceived 

workload. 

5.8.4. 1  Praotioe  and  Decision  Quality. 


H4a.  The  quality  of  team  decisions  In  the  TRAP  will  improve 
with  practice,  especially  in  early  sessions. 

The  results  of  this  xtudy  suggest  that  praatice  does  affect  overall 
decision  quality  on  the  TRAP.  Using  the  ratio  of  team  score  to  model 
score  as  a  measure,  teams  performed  significantly  better  in  sessions  3 
and  4  (75.0  percent  of  model)  than  in  sessions  1  (70,7  percent  of  model) 
and  2  (72.7  percent  of  model)  (p  ■  .0003). 

Using  the  ratio  of  a  team's  points  earned  from  certain-valued 
events  to  the  team's  total  points  as  a  behavioral  measure,  teams  earned 
a  higher  proportion  of  their  points  from  certain-valued  events  in 
session  2  (53.6  percent)  than  in  any  other  session  (session  1,  40. 8 
percent;  session  3,  40.6  percent;  session  4,  47.4  percent)  (p  ■  ,0083). 
For  comparison,  the  model  earned  46.7  percent  of  its  points  from 
certain-valued  events;  after  deviating  in  session  2,  teams  returned  to  a 
more  appropriate  proportion  during  sessions  3  and  4, 
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In  terms  of  the  proportions  of  specific  certain-valued  events 
completed,  praatioe  had  a  main  effeot  only  on  the  proportion  of  2- 
player-4-points-per-player  events  completed.  Teams  completed  more  of 
these  events  in  session  2  (44.3  percent)  than  in  any  other  session  (38.1 
percent  in  session  1,  30.2  percent  in  session  3,  and  38.0  percent  in 
session  4)  <p  ■  .0310).  For  the  sake  of  comparison,  the  model  completed 
58.0  percent  overall  with  essentially  no  variation  among  sessions). 

Practice  also  affeoted  information  seeking  activity.  Teams  sought 
additional  information  about  significantly  more  uncertain  events  in 
session  4  (51.2  percent)  than  in  either  session  1  (47.7  percent)  or 

session  2  (43.9  percent)  (p  ■  .0112).  For  comparison,  the  computer 

model  sought  Information  about  03.7  percent  of  the  uncertain  events. 

Hypothesis  H4a,  then,  was  supported  by  the  results  of  this  study. 
The  findings  supported  the  expected  Improvement  in  decision  quality  over 
the  four  experimental  sessions.  The  results  for  the  behavioral 
variables  suggest  that  the  performance  improvement  was  achieved  by  first 
improving  efficiency  in  handling  certain-valued  events  and  then  by  more 
appropriately  handling  uncertain  events. 

5.8. 4. 2  Fractice  and  Perceived  Workload 

H4b.  The  perceived  workload  in  the  TRAP  will  decline  with 

practice,  especially  in  early  sessions. 

Using  SWAT  ratings  as  a  measure,  team  perceptions  of  workload  were 
significantly  lower  in  session  3  (33.5)  than  in  any  other  session 
(session  1,  39.5;  session  2,  42.1;  session  4,  40.2)  (p  ■  .0001), 

Hypothesis  H4b,  then,  is  not  clearly  supported  by  the  results  of  this 
study;  variations  in  perceived  workload  do  not  match  the  expected 
pattern. 

5.8.5  Interactive  Effects  of  Time  Stress  and  Information  Presentation 
Form 

This  research  assesses  the  interaction  between  time  stress  and 
Information  presentation  form,  The  following  two  subsections  repeat  the 
hypotheses  and  present  findings  concerning  the  impact  of  the  interaction 
on  decision  quality  and  peraeived  workload. 

5.8.5. 1  Time  Stress/Information  Presentation  Form  and  Decision  Quality. 

H5a.  Time  pressure  and  presentation  treatment  will  interact 

such  that  the  advantage  of  single-featured  coding  over 
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conjunctiva  coding  jor  deaision  quality  will  ba  greater  under 
high  time  pressure  than  under  moderate  time  pressure. 
However!  the  relative  eff eotiveneif  of  alphabetic  and  color 
coding  for  decision  quality  will  remain  the  same  whether  under 
high  or  moderate  time  pressure. 

The  results  of  this  Study  suggest  that  there  are  no  interaction 
effects  between  time  stress  and  information  presentation  form  on 
decision  quality.  Using  the  ratio  of  team  soore  to  model  score  as  a 
measure,  variations  in  time  pressure  did  not  modify  the  lack  of  effects 
related  to  the  coding  schemes  (p  *  .8013).  There  were  also  no 
interaction  effects  between  time  stress  and  information  presentation 
form  using  the  ratio  of  a  team's  points  earned  from  certain-valued 
events  to  the  team's  total  points  as  a  behavioral  measure  (p  «  .0920). 

Hypothesis  H9a,  then,  was  partially  supported  by  the  results  of 
this  study.  As  expected,  time  pressure  did  not  modify  the  relative 
effectiveness  of  alphabetio  and  color  coding  schemes  for  decision 
quality.  However,  contrary  to  expectations,  time  pressure  also  failed 
to  modify  the  relative  effectiveness  of  single- featured  and  oonjunotive 
coding  sahemes  for  decision  quality. 

5 . 8 . 5 . 2  Time  StreSS/lnf ormation  Presentation  Form  and  Perceived 
Workload . 

HSb,  Time  pressure  and  praientatlon  treatment  will  interact 
suoh  that  the  advantage  of  single-featured  coding  over 
conjunctive  coding  for  perceived  workload  will  be  greater 
under  high  time  pressure  than  under  moderate  time  pressure, 
However,  the  relative  Impact  of  alphabetio  and  color  aoding  on 
perceived  workload  will  remain  the  same  whether  under  high  or 
moderate  time  pressure. 

Using  SWAT  ratings  as  a  measure  of  perceived  workload,  there  were 
no  interaction  effects  between  time  stress  and  information  presentation 
form  (p  •  .2924).  Hypothesis  H5b,  then,  was  partially  supported  by  the 
results  of  this  study.  As  expected,  time  pressure  did  not  modify  the 
relative  impact  of  alphabetic  and  color  coding.  However,  contrary  to 
expectations,  time  pressure  also  failed  to  modify  the  relative  impact  of 
single-featured  and  conjunctive  coding  on  perceived  workload. 

5.8.8  Interactive  Effects  of  Practice  and  Information  Presentation  Form 


This  research  assessed  the  interaction  between  practice  and 
information  presentation  form.  The  following  two  subsections  repeat  the 


hypotheses  and  present  findings  concerning  the  impact  of  the  interaction 
on  decision  quality  and  perceived  workload. 

5.8.6. 1  Practlce/lnformatlon  Presentation  Form  and  Decision  Quality. 

H6a.  Practice  and  presentation  treatment  will  interact  such 
that  any  differences  in  the  relative  effectiveness  of 
alphabetic  and  color  coding  for  deoision  quality  will 
attenuate  over  the  four  experimental  sessions.  However, 
differences  in  the  relative  effectiveness  of  oonjunotive  and 
single-featured  coding  will  remain  constant  over  the  four 
experimental  sessions. 

The  results  of  this  study  suggest  that  there  are  no  interaction 
effects  between  praotioe  and  information  presentation  form  on  decision 
quality.  Using  the  ratio  of  team  Soore  to  model  Soore  as  a  measure, 
practice  did  not  modify  the  lack  of  effects  related  to  the  coding 
schemes  (p  ■  .0112).  There  were  also  no  interaction  effects  between 
practice  and  information  presentation  form  using  the  ratio  of  a  team's 
points  earned  from  certain-valued  events  to  the  team's  total  poirits  as  a 
behavioral  measure  (p  >  ,2708).  '  * 

Hypothesis  H8a,  then,  was  partially  supported  by  the  results  of 
this  study.  As  expeated,  praotioe  did  not  modify  the  relative 
ef feotiveness  of  conjunctive  and  single-featured  ooding  schemes  for 
decision  quality.  However,  contrary  to  expectations,  praotioe  also 
failed  to  modify  the  relative  effectiveness  of  alphabetic  and  color 
coding  schemes  for  decision  quality. 

5. 8. <3. 2  Praotlce/Informatlon  Presentation  Form  and  Perceived  Workload, 

H8b.  Practice  and  presentation  treatment  will  interact  such 
that  any  differences  in  perceived  workload  associated  with 
alphabetic  and  color  ooding  will  attenuate  over  the  four 
experimental  sessions.  However,  differences  in  perceived 
workload  associated  with  conjunctive  and  single-featured 
coding  will  remain  constant  over  the  four  experimental 
sessions . 

Using  SWAT  ratings  as  a  measure  of  perceived  workload,  there  were 
no  interaction  effects  between  practice  and  information  presentation 
form  (p  *  .8418).  Hypothesis  H9b,  then,  was  partially  Supported  by  the 
results  of  this  study.  As  expected,  practice  did  not  modify  the 


relative  impact  of  alphabetic  and  color  ooding  on  perceived  workload. 
However,  contrary  to  expectations,  practice  also  failed  to  modify  the 
relative  impact  of  single-featured  and  conjunctive  coding. 


5.8.7  Interactive  Effects  of  Tima  Strega  and  Suggested  Decision 
Strategy 

Thii  research  assessed  the  interaction  between  time  stress  and  the 
suggested  decision  strategy.  The  following  two  subsections  repeat  the 
hypotheses  and  present  findings  concerning  the  impact  of  the  interaotion 
on  decision  quality  and  peraeived  workload. 

5.8.7. 1  Time  Stress/Suggested  Decision  Strategy  and  Decision  Quality. 

H7a.  Time  pressure  and  heuristics  treatment  will  interact 
.such  that  the  difference  in  decision  quality  related  to 
heuristics  treatment  will  be  greater  under  high  time  pressure 
than  under  moderate  time  pressure. 

The  results  of  this  study  suggest  that  time  stress  and  the 
suggested  decision  strategy  do  interact.  Using  the  ratio  of  tram  score 
to  model  score  as  a  measure,  the  heuristics  treatment  vid  time  pressure 
interacted  such  that,  under  high  time  pressure,  tuams  with  i.euristlcs 
(S8.0  percent  of  model)  did  little  better  than  teams  without  heuristics 
(05. 7  percent  of  model);  however,  under  moderate  time  pressure1,  teams 
with  heuristics  scored  Significantly  higher  (82.4  percent  of  model)  than 
did  teams  without  heuristics  (76.6  percent  of  model)  (p  *  .0300). 

Using  the  proportion  of  2-player-3-points-per-player  event 
completed  as  a  behavioral  measure,  there  was  a  three-way  interaction 
among  the  heuristics,  time  stress,  and  praotice  variables  (p  •  .0036), 
Table  5.16  provides  the  proportions  of  these  events  completed  by  teams 
with  and  without  heuristics  during  each  session  under  high  and  moderate 
time  stress.  For  comparison,  the  model  completed  about  4.3  peraent  of 
the  4-point  events.  Figure  5.2  graphically  illustrates  the  interaction. 
Except  during  session  1  under  moderate  time  pressure  (when  teams  with  or 
without  heuristics  completed  about  the  same  number  of  4-point  events)  , 
teams  with  heurlstios  consistently  completed  a  very  low  proportion  of  4- 
point  events,  The  highest  proportions  of  4-point  events  completed  by 
teams  without  heuristics  were  in  sessions  2  and  3  under  moderate  time 
stress;  the  lowest  proportion  for  teams  without  heuristics  was  in 
session  3  under  high  time  stress.  The  result)  suggest  that  teams  with 
heuristics  learned  quiokly  to  ignore  4-point  events;  beyond  session  1, 
time  stress  had  little  impact  on  the  proportion  completed.  Except  under 
moderate  time  pressure  in  session  1,  teams  without  heuristics 
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consistently  completed  a  higher  proportion  oi  4-point  events  than  did 
teams  with  heuristios;  the  magnitude  oi  the  difierenoe  varied  depending 
on  both  practice  and  time  stress,  Under  high  time  stress ,  teams  without 
heuristios  completed  a  moderate  number  of  4-point  events  except  for  a 
lower  proportion  in  session  3s  under  moderate  time  stress,  teams  without 
heuristics  completed  a  moderate  number  of  4-point  events  in  sessions  1 
and  4  and  a  higher  proportion  in  sessions  2  and  3. 

TABLE  8.18  Time*aess*heur  Effects  on  Completing  4-Foinb  Events 

Session 


1 

2 

3 

4 

With  heuristics 

High  stress 

0.3X 

0,7% 

1.0X 

l.OX 

Moderate  stress 

3, 8X 

1.0X 

1.7X 

1.7X 

Without  heuristics 

High  stress 

wgm 

4.8X 

IQ 

Moderate  stress 

K89 

7.7% 

III 

Hypothesis  H7a,  then,  was  not  supported  by  the  results  of  this 
study,  The  interaction  between  time  stress  and  the  suggested  decision 
strategy  did  affect  decision  quality,  but  not  in  the  expected  direction, 
The  suggested  decision  strategy  had  a  greater  impact  on  overall 
performance  under  moderate  time  pressure  than  under  high  time  pressure. 
The  speaifia  behavioral  impact  on  completing  4-polnt  events  depended  on 
practice, 

8. 8. 7. 2  Time  Strsaa/SuMested  Decision  Strategy  and  Perceived  Workload. 

H7b.  Time  pressure  and  heuristios  treatment  will  interaot 
such  that  the  difierenoe  in  peroeived  workload  related  to 
heuristios  treatment  will  be  greater  under  high  time  pressure 
than  under  moderate  time  pressure, 

Using  SWAT  ratings  as  a  measure,  the  Interaction  between  time 
stress  and  the  suggested  decision  Strategy  had  no  Significant  effects  on 
perceived  workload  (p  ■  .4490).  Hypothesis  H7b,  then,  was  not  supported 
by  the  results  oi  this  study;  time  pressure  did  not  modify  the  impact  of 
the  suggested  decision  strategy  on  peroeived  workload, 


90 


Time *8e8S*h©ur  interaction  Plot 
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5 . S . 8  Interactive  Effects  of  Practice  and  Suggested  Decision  Strategy 
Thii  research  uitaid  the  interaction  between  practiae  and  the 
suggested  decision  strategy,  Tha  following  two  subsections  repeat  the 
hypotheses  and  preaant  findinga  oonoerning  tha  impact  of  tha  interaction 
on  deaiaion  quality  and  perceived  workload. 

5 . 8 . 8 . 1  Fractlce/Suaaeated  Daclalon  Strategy  and  Paolalon  Quality. 

.  H8a.  Practice  and  tha  auggaatad  daoiaion  strategy  will 
interact  auoh  that  tha  advantage  with  proviaion  and  diaouaaion 
of  daoiaion  hauriatici  will  attenuate  over  the  four 
experimental  sessions. 

The  results  of  this  study  suggest  that  there  are  no  interaction 
effects  between  practice  and  the  suggested  decision  strategy  on  daoiaion 
quality  except  for  the  three-way  heur*time«seas  interaction  effeot  on 
completing  2-player-2-points-per-player  events  (See  Seotion  B.8.6.1). 
Using  the  ratio  of  team  score  to  model  score  as  a  measure,  practice  and 
the  heuristics  treatment  did  not  interact  (p  ■  .8188).  There  Was  also 

i 

no  interaction  between  praatiea  and  the  suggested  decision  Strategy 

using  the  ratio  of  a  team's  points  earned  from  certain-valued  events  to 

the  team's  total  points  as  a  behavioral  measure  (p  ■  .0004).  Hypothesis 

H8a ,  then,  was  not  Supported  by  the  results  of  this  study.  In  other 

words,  the  heuristics  were  effective  immediately,  and  teams  with 

heuristics  retained  their  advantage  throughout  the  experiment. 

5 .0 .8.2  Praotloe/Buggested  Decision  Strategy  and  Perceived  Workloa<|. 

H8b.  Praotice  and  the  suggested  decision  strategy  will 
interact  suah  that  differences  in  perceived  workload  between 
teams  with  and  those  without  provision  and  discussion  of 
dealsion  heuristics  will  attenuate  over  the  four  experimental 
sessions. 

Using  SWAT  ratings  as  a  measure  of  perceived  workload,  there  werr 
no  interaction  effects  between  praotice  and  the  suggested  decision 
strategy  (p  ■  ,3892).  Hypothesis  H8b,  then,  was  not  supported  by  the 
results  of  this  study;  practice  did  not  modify  the  impact  of  the 
suggested  decision  strategy  on  perceived  workload. 

S . 8 . 0  Relative  Effects  of  Information  Presentation  Form  and  Suggested 
Decision  Strategy 

This  research  compared  the  impact  of  information  presentation  form 


and  the  suggested  decision  strategy.  The  following  two  subsections 


repeat  the  hypotheses  and  present  finding*  concerning  the  relative 
effects  of  the  two  variables  on  decision  quality  and  perceived  workload. 
8.0.9. 1  Information  Presentation  Porm  vs.  Sukkested  Decision  Strategy 
and  Decision  Quality. 

H9a,  The  magnitude  of  impact  on  decision  quality  related  to 
information  presentation  form  and  to  the  suggested  decision 
strategy  will  be  the  same. 

The  most  important  finding  of  this  study  is  that,  although  no 
significant  effects  of  information  presentation  form  on  decision  quality 
were  detected,  heuristics  did  positively  affect  decision  quality  in  a 
dynamic  group  intellective  task  involving  uncertainty.  Therefore, 
hypothesis  H9a  concerning  the  relative  effects  of  the  two  treatments  on 
decision  quality  was  not  supported;  the  suggested  decision  strategy  had 
a  greater  impact  on  decision  quality  than  did  the  information 
presentation  form. 

8. 0.9. 2  Information  Presentation  Form  v*.  BukkeSted  Decision  ktrateky 
and  Perceived  Workload. 

H9b.  The  magnitude  of  impact  on  perceived  workload  related  to 
information  presentation  form  and  to  the  suggested  decision 
strategy  will  be  the  same. 

No  significant  effects  of  either  information  presentation  form  or 
the  suggested  decision  strategy  on  perceived  workload  were  detected. 
Therefore,  hypothesis  H9b  concerning  the  relative  effects  of  the  two 
treatments  on  perceived  workload  was  supported. 

8.9  Summary  of  Results 

The  most  Important  finding  of  this  study  was  that,  in  a  dynamic 

group  choice  task,  heuristics  had  a  much  greater  positive  lmpaat  on 

decision  quality  than  did  varying  the  coding  scheme  by  which  decision 

data  was  presented.  Other  findings  include  the  following; 

Neither  Information  presentation  form  nor  heuristics  had  an 
lmpaat  on  SWAT  measures  of  perceived  workload. 

Groups  performed  better  and  perceived  lower  levels  of  workload 
under  moderate  time  pressure  than  under  high  time  pressure. 


Groups  performed  better  and  perceived  lower  levels  of  workload 
with  practice. 


Neither  time  stress  nor  practice  modified  the  finding  that 
information  presentation  form  had  no  effect  on  either  decision 
quality  or  perceived  workload. 

Heuristics  had  a  greater  effect  on  decision  quality  under 
moderate  time  stress  than  under  high  time  stress,  However, 
time  stress  did  not  modify  the  finding  that  heuristics  had  no 
effect  on  perceived  workload. 

--  ,  Practice  did  not  change  the  effects  of  heuristics  on  decision 
quality  nor  did  it  modify  the  finding  that  heuristios  had  no 
effect  on  perceived  workload. 

This  chapter  presented  a  statistical  analysis  of  the  results  of  the 
research.  T1  next  ohapter  places  the  findings  in  the  context  of  the 
conceptual  model  and  the  relevant  literature  and  discusses  research 
implications  of  the  results. 
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0,0  DISCUSSION  OF  RESULTS  AND  CONCLUSIONS 

The  first  section  of  this  chapter  presents  a  discussion  of  the 
roaearch  findings,  Ths  discussion  is  followed  by  a  summary  of  the 
overall  conclusions  from  this  rsstsrch. 

0 . r  Discussion  of  Results 

This  station  optns  with  s  discussion  of  ths  most  important  result 
of  Ui«  study(  which  concerns  the  relative  impact  on  decision  quality  of 
decision  process  aids  and  information  presentation  forms.  The  opening 
subsection  is  followed  by  a  discussion  of  other  findings.  A  third 
subsection  briefly  presents  some  observational  findings  not  dearly 
related  to  any  of  the  hypotheses. 

6.1.1  Discussion  of  Primary  Result 

The  most  important  finding  of  this  research  is  that,  in  a  dynamic 
group  intslleativs  task,  ths  impact  on  decision  quality  of  aids  whiah 
guided  the  decision  proosSS  was  much  greater  than  the  impact  of  the  form 
in  which  information  was  presented.  In  the  context  of  the  conceptual 
model  for  this  study,  diteat  Interventions  in  ths  decision  making 
process  by  means  of  a  suggested  decision  strategy  had  a  graater  ultimate 
impact  on  decision  quality  than  did  indirect  interventions  by  means  of 
modifications  to  the  information  presentation  form. 

0. 1.1.1  Suggested  Decision  Strategy  and  Decision  Quality.  To  better 
understand  why  teams  with  heuristics  performed  better  than  those  without 
heuristics,  videotapes  of  six  teams  (ths  top  three  and  ths  bottom  three 
teams  selected  on  ths  basis  of  overall  average  ratios  of  team  scores  to 
model  scores)  were  reviewed.  None  of  the  bottom  three  teams  recsivsd 
the  heuristics;  ths  top  two  teams  rsocived  the  heuristias,  and  the  third 
best  team  quickly  developed  and  adopted  a  strategy  very  similar  to  that 
proposed  in  the  heuristics.  Contrasts  betwesn  ths  top  three  and  the 
bottom  three  teams,  then,  will  be  used  to  illustrate  decision-making 
differences  between  teams  using  strategies  oriented  toward  the 
heuristics  and  those  using  alternate  strategies. 

By  providing  and  encouraging  discussion  of  the  heuristics,  it  was 
expected  that  explioit  attention  paid  to  an  experience-based  strategy 
would  increase  the  probability  that  team  members  would  recognise  correct 
choices  in  the  TRAP  and  would  thereby  Improve  decision  quality.  Three 
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comparisons  between  the  top  three  teams  and  the  bottom  three  teams 
sugg*  t  that  teams  v.lth  the  heuristic*  (or  with  a  clear  commitment  to  a 
heuristics-like  Strategy)  were  better  prepared  to  recognise  correct 
choices  than  were  teams  without  the  heuristics.  First,  the  top  three 
teams  wasted  less  time  searching  for  alternatives  (an  average  of  10.0 
time  units  per  trial  when  no  resources  were  committed)  than  did  the 
bottom  three  teams  (IS. 6  tims  units  psr  trial),  Second,  the  top  three 
teams  made  poor  choices  (selsctsd  less  than  the  best  combination  of 
events)  less  frequently  (an  average  of  3.0  times  psr  trial)  than  did  the 
bottom  three  teams  (8,7  times  psr  trial).  Finally,  the  top  three  teams 
reversed  commitments  (used  their  reset  buttons)  less  frequently  (an 
average  of  1.0  times  psr  trial)  than  did  the  bottom  three  teams  (3.0 
times  per  trial) . 

The  link  between  the  decision  heuristics  and  improvsc  performance 
was  also  hypothesized  in  part  on  ths  basis  of  providing  ths  heuristics 
as  a  technique  for  reducing  human  bias  in  dsaision  making  (FISC82). 
Each  of  the  four  heuristics  provided  was  intended  to  overcome  a  specific 
bias  evident  in  ths  performance  of  previous  TRAP  participants,  Behavior 
differences  related  to  each  of  the  expected  biases  between  teams  with 
heuristic*  and  those  without  heuristics  help  to  oxplain  the  overall' 
performance  differences. 

0.1. 1.1.1  Heuristic  *1.  Ths  first  hsuristic  snoouragsd  teams  to 
complete  high-certain-valued  events  as  a  first  priority.  Spool fically , 
teams  were  told  to  look  for  both  3-person-S-polnte-psr-psrSon  events  and 
2-person-4-polnt*-per-psr*on  events  in  combination  with  a  S-point  event 
for  the  third  person.  All  teams,  with  or  without  heuristics,  complete 
all  or  nearly  all  of  the  3-per*on-0-point*-per-per*on  events.  There  was 
also  no  statistically  significant  difference  between  teams  with 
heuristics  and  those  without  in  the  number  of  2-per*on-4-point*-per- 
person  events  completed.  Review*  of  ths  videotapes ,  however,  suggest 
that  teams  using  a  heuristics-orisntsd  strategy  tended  to  be  mors 
selective  in  completing  these  events. 

Teams  without  heuristics  completed  43.0  percent  of  the  total  number 
of  2-person-4*points-psr-person  events;  ths  bottom  three  teams  in 
particular  completed  91.2  percent.  Of  the  available  combinations  of  8- 


point  events  with  5-point  events  for  the  third  person,  th*  bottom  three 
team/)  still  completed  51.2  percent.  Judging  by  their  comments  end 
actions,  the  bottom  three  teams  Simply  sought  to  complete  a  significant 
number  of  the  2-person-4-points-per-per*on  events.  Where  high-value 
combinations  were  completed,  they  wore  often  discovered  only  because  the 
third  person  went  looking  for  a  one-person  event  after  the  other  team 
members  indicated  intent  to  commit  resources  to  the  two-person  event. 

By  comparison,  teams  with  heuristics  completed  37.1  percent  of  the 
total  number  of  2-person-4-pointS-per*perSon  events;  the  top  three  teams 
in  particular  completed  about  the  same  number  (37.2  percent).  However, 
of  the  available  combinations  of  6-point  events  with  5-point  events  for 
the  third  person,  the  top  three  teams  completed  62.2  percent.  Judging  by 
their  comments  and  actions,  the  top  three  teams  looked  for  high-value 
combinations  involving  2-pirson-4-points-per-person  events. 

6. 1.1. 1.2  Heuristic  »2.  Of  the  heuristics  provided,  the  one  that 
seemed  to  have  the  greatest  overall  impact  was  the  second  heuristic 
which  guided  teams  toward  using  expected  utility  theory  in  dealing  with 
uncertain  events.  As  suggested  by  prospect  theory  [KAHN79],  teams 
operating  without  the  heuristics  in  this  study  valued  certain  gains  over 
merely  probable  gains,  Specifically,  teams  without  heuristics  earned  a 
larger  proportion  of  their  points  from  certain-valued  events  than  from 
uncertain  events  and  sought  additional  information  about  fewer  uncertain 
events  than  did  teams  with  heuristics.  Tsams  operating  with  the 
heuristics,  on  the  other  hand,  handled  uncertain  events  according  to 
expected  utility  theory  CKAHN79],  illustrating  the  debiasing  effects  ol 
the  heuristics  [FISC82], 

The  magnitude  of  the  impact  of  the  Second  heuristic  seemed  to  be 
related  to  two  lectors.  First,  20  uncertain  events  represent  a 
significant  proportion  (31  percent)  of  the  available  events  in  eaoh 
trial,  so  differences  in  strategy  concerning  uncertain  events  resulted 
in  clear  differences  in  performance.  Second,  differences  in  strategy 
concerning  uncertain  events  betwesn  teams  with  heuristics  and  thoss 
without  were  more  obvious  and  consistsnt  than  were  differences  In 
strategy  related  to  any  of  the  other  heuristies.  Teams  with  tha 
heuristics  very  consistently  followed  the  expected  utility  guidelines 
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for  handling  uncertain  events.  By  contraat,  teami  without  the 
heurifitica  ranged  from  almoat  totally  avoiding  uncertain  eventa  to 
depending  almoat  exclusively  on  uncertain  eventa.  Teama  without  the 
heuriatica  alao  varied  greatly  in  their  aelectivity  in  committing 
reaourcea  to  uncertain  eventa  aubaequent  to  aeeking  additional 
information.  For  example,  among  the  bottom  three  teama,  one  team 
committed  reaourcea  indiecriminately  to  almoat  every  uncertain  event 
about  which  it  Bought  information,  one  team  eaaentially  ignored 
uncertain  eventa  until  the  lait  two  triala,  and  one  team  committed 
reaourcea  to  uncertain  eventa  on  the  baaia  of  high  probability  of  payoff 
rather  than  high  expeated  value. 

6. 1.1. 1.3  Heurlatlc  *3.  The  third  heuriatic  encourae«d  teama  to  Ignore 
low-certain-valued  eventa.  Aa  expeated,  teama  with  heuriatica  completed 
a  atatiatically  aignificant  lower  number  of  low-valued  eventa  than  did 
teama  without  heuriatica.  The  bottom  three  teama  completed  more  than 
three  timea  aa  many  low-valued  eventa  aa  did  the  top  three  teama.  divan 
the  email  proportion  of  low-valued  eventa  completed  even  by  teama 
without  heuriatica,  however,  the  direct  impaot  of  thia  heuriatic  waa 
limited.  The  average  difference  in  pointe  earned  from  low-valued  eventa 
between  teama  with  heuristics  and  thoae  without  waa  leaa  than  1.1  pointa 
per  trial.  An  indirect  Impaot  of  the  heuriatic  waa  that  teama  with 
heuriatica  committed  reaourcea  to  more  valuable  eventa  while  teama 
without  heuriatioa  were  committed  to  earning  that  1  point  per  trial,  but 
there  la  no  dear  mechaniam  for  placing  a  value  on  the  indirect  benefit. 

8.1. 1.1.4  Heuriatic  M.  The  fourth  heuriatic  auggeated  that  teama 
remain  aynchronited  in  committing  reaourcea  to  eventa.  In  other  worda, 
team  membera  ahould  commit  individual  reaourcea  to  a  combination  of 
eventa  aimultaneoualy  ao  that  all  team  reaouraea  will  be  free  for 
follow-on  commitmenta  aimultaneoualy  and  at  regular  intervale. 
Synchronization  ahould  reduce  conflicting  demanda  on  a  reaource  and, 
therefore,  aimplify  the  deoieion-making  proceed.  In  reviewing  the 
videotaped,  the  occurrence  of  a  load  of  synchronization  waa  defined  to 
be  any  occasion  on  whioh  e  team  member  committed  hia/her  resource  two 
time  units  or  more  later  than  other  reaouroe  commitmenta  by  the  team. 


OB 


On  average,  the  top  three  teams  loit  synchronieation  2.2  times  par  trial 
while  the  bottom  three  teams  lost  aynchroniaation  5.9  times  par  trial. 
0.1. 1.2  Interactive  Effect*  of  Tima  Stress  and  Suggested  Decision 
Strategy.  Heuristics  had  a  greater  effeot  on  daoiaion  quality  undar 
moderate  time  atraaa  than  undar  high  time  atrtaa.  Using  tha  overall 

performanaa  maaaura  (ratio  of  team  aoora  to  aomputer  modal  aoora)  ,  tha 
hauriatica  had  a  politiva  influenoe  on  daoiaion  quality  only  undar 
moderate  time  preaaure.  Two  intarpratationa  of  thia  finding  ware 
conaidarad. 

Firat,  it  could  be  that,  given  time  to  apply  tha  heuriatioa,  tha 
hauriatica  provide  a  performance  gain  with  no  apparent  coat  in  paroalvad 
workload.  In  other  worda,  undar  high  time  atraaa,  taama  may  hava 
raaortad  to  vary  aimpla  daoiaion  ruled,  whether  or  not  they  had  received 
the'  hauriatica,  reacting  aa  beat  they  could.  Undar  moderate  time 
preaaure,  toama  had  time  to  more  oarefully  conaidar  thair  altarnativaa 
and  could  adopt  more  complex  daoiaion  ruled,  whether  tha  daciaion  ruled 
ware  provided  externally  or  oraatad  internally.  Tha  heuriatioa,  then, 
may  hava  focuaad  team  attention  on  applying  a  conaiatantly  'good*  aet  of 
daciaion  ruled  that  improved  performance  without  making  the  taak  any 
more  difficult  than  it  waa  for  taama  who  applied  thair  own  daciaion 
ruled. 

If  the  lack  of  algnlf leant  impact  of  tha  heuriatioa  on  SWAT  ratingd 
la  aaaumad  to  be  due  either  to  counteracting  affect a  or  to  meaauramant 
problama  with  SWAT,  a  aecond  interpretation  la  that  dome  of  the 
heuristics  may  hava  added  to  tha  demanda  on  human  information  proceeding 
rather  than  routinicing  declaiona.  It  could  bo,  aa  for  tha  firat 
interpretation,  that  undar  high  time  preaaure,  taama  raaortad  to  vary 
simple ,  reactive  strategies;  teams  with  hauriatica,  whose  information 
processing  capacity  may  already  have  been  taxed  by  the  demands  of  the 
high  spaed  taak,  tailed  to  apply  tha  whole  aet  of  heuristics.  Under 
moderate  time  pressure,  however,  teams  with  the  heuristics  may  have  had 
greater  capacity  to  consider  and  to  apply  the  hauriatica  to  thair 
decisions . 

Both  statistical  and  observational  findings  along  with  skepticism 
about  the  SWAT  ratings  lead  to  favoring  the  aecond  interpretation.  For 
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example,  3inca  there  were  no  statistically  significant  effects  of  time 
pressure  or  of  any  interaction  involving  time  pressure  on  resource 
commitments  to  uncertain  events,  it  could  be  that  the  heuristic 
concerning  handling  of  uncertain  events  was  particularly  easy  to  apply. 
If  so,  it  may  have  routiniaed  djeisions  about  those  events  and  become 
equally  useful  under  high  or  moderate  time  pressure. 

On  the  other  hand,  observations  of  the  best  three  and  the  poorest 
three  teams  suggest  that  applying  the  heuristic  to  look  for  high-value 
combinations  of  events  involving  3-perion-4-poinbs-per-person  events  may 
have  been  more  difficult,  Poor  teams  (all  operating  without  heuristics) 
completed  more  8-point  events  under  moderate  time  pressure,  when  they 
had  more  time  to  choose  events,  than  under  high  time  pressure.  For  some 
of  those  8-polnt  events  (about  the  same  proportion  as  under  high  time 
pressure) ,  the  third  person  found  a  8-point  event  to  complete  once  team 
members  had  chosen  to  work  the  8-point  events.  Therefore,  their 
increased  rate  of  completing  high-value  combinations  simply  matched 
their  increased  rate  of  completing  8-polnt  events.  The  best  teams  (all 
operating  either  with  the  heuristics  or  with  a  heuristics-like 
strategy) ,  completed  the  same  number  of  8-polnt  events  but  more  high- 
value  combinations  under  moderate  time  pressure  than  under  high  time 
pressure.  Combinations  of  events  wsre  apparently  more  difficult  to 
Identify  than  single  events.  With  more  time  to  search  for  high-value 
combinations,  teams  with  heuristics-oriented  strategies  tended  to 
complete  more  high-value  combinations  under  moderate  time  pressure  than 
under  high  time  pressure.  Since  applying  the  heuristic  on  high-value 
combinations  seemed  difficult,  applying  it  may  have  exceeded  information 
processing  capacity  at  the  high  time  stress  level  but  not  at  the 
moderate  level. 

Finally,  the  seoond  interpretation  attributes  ths  lack  of  impact  of 
the  heuristics  on  perceived  workload  to  one  or  both  of  two  factors.  The 
roufcinizing  effect  of  one  or  more  of  the  heuristics  may  have  been 
counteracted  by  the  difficulty  in  applying  other  heuristics,  resulting 
in  no  net  effect  on  perceived  workload.  The  second  factor  is  that  SWAT 
ratings  may  not  have  adequately  captured  perceived  workload. 


If  the  second  interpretation  is  correct,  there  are  interesting 
implications  for  supporting  dynamia  intellective  tasks  in  practice 
tpertups  some  monitoring  tasks  associated  with  air  traffic  control  or 
aircraft  cockpit  operations)  that  seem  worthy  of  exploration.  Up  to 
some  undetermined  level  of  stress,  providing  decision  rules  to  human 
decision  makers  may  be  sufficient  to  assure  good  performance.  Beyond 
the  point  at  which  applying  the  deoision  rules  exoeeds  information 
processing  capacity,  however,  other  mechanisms,  euoh  as  invoking  expert 
system  support,  may  be  more  effective. 

In  future  studies,  the  impact  of  the  heuristias  themselves  on 
information  processing  capacity  should  be  assessed.  The  effects  of 
variations  in  the  scope  of  the  heuristics  and  in  the  form  in  which  the 
heuristics  are  provided  would  help  to  explain  the  interactive  effects  of 
time  stress  and  heuristics. 

e. 1.1.3  Interactive  Effects  of.  Practice  and  Suggested  Decision 
Strategy.  Practice  did  not  change  tha  effects  of  heuristics  on  deoision 
quality  nor  did  it  modify  the  finding  that  heuristics  had  no  effect  on 
perceived  workload,  Although  the  results  of  this  study  failed  to 
demonstrate  statistically  that  practice  could  attenuate  differences  in 
decision  quality  between  teams  with  heuristios  and  those  without, 
subjective  evidence  suggested  that  some  teams  without  the  heuristios 
moved,  at  different  rates,  toward  a  heuristics-oriented  strategy,  The 
one  team  in  the  top  three  that  did  not  have  the  heuristics  adopted  a 
similar  strategy  early  and  continued  to  evolve  toward  a  strategy  that 
closely  matched  the  heuristios.  They  completed  a  high  proportion  of 
htgh-aertain-valued  events  and  combinations  of  events,  avoided  low¬ 
valued  events,  and  remained  reasonably  well  synchronised.  They  also 
gradually  increased  the  number  of  uncertain  events  about  which  they 
sought  additional  information  and  became  increasingly  selective  about 
committing  resources  to  uncertain  events  with  high  expected  value. 

One  of  the  bottom  three  teams  abruptly  increased  attention  to 
uncertain  events  in  the  last  two  trials  and  realised  obvious 
Improvements  in  their  scores.  Given  additional  trials,  they  may  also 
have  further  refined  their  Strategy  to  approach  the  heuristics.  Other 
teams  without  the  heuristics  (including  the  other  two  bottom  teams) 
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seemed  to  settle  early  on  a  strategy  and  to  maintain  it  throughout  the 
experiment.  Research  about  how  and  why  some  groups  choose  to  Innovate 
while  others  maintain  initial  strategies  would  be  useful. 

6. 1.1.4  Information  Presentation  Form  and  Decision  Quality.  The  laok 
of  a  statistically  significant  effect  of  Information  presentation  form 
or  of  any  interaction  involving  information  presentation  form  on 
decision  quality  in  the  TRAP  is  an  important  finding.  It  reinforces 
findings  that  humans  adapt  readily  to  and  perform  consistently  under  a 
variety  of  task  conditions  tLEMASC). 

The  laok  of  statistically  significant  performance  differences 
between  groups  using  Single-featured  coding  schemes  and  those  using 
conjunctive  schemes  should  not  be  taken  as  evidence  against  feature 
integration  theory.  Tests  of  the  theory  typically  present  objects  only 
until  participants  maka  a  response  (less  than  3  seconds  for  displays 
with  density  comparable  to  the  TRAP  displays)  CTRBI80 ) .  The  intent  of 
attention  researoh  is  to  detect  differences  in  the  initial  mental 
processing  of  the  objeat.  TRAP  events,  by  contrast,  rtmain  visible  and 
available  for  initiating  rasourot  commitments  for  up  to  30  seconds  under 
high  time  stress  and  up  to  40  Second!  under  moderate  time  stress.  The 
relative  longevity  of  TRAP  events  provides  the  time  for  confirmation  of 
the  identity  of  every  object.  Any  disadvantage  in  Initial  mental 
processing  of  TRAP  avsnts  presented  via  any  particular  coding  scheme  may 
be  compensated  for  by  the  opportunity  to  confirm  the  location  and 
identity  of  the  events. 

The  lack  of  performance  differences  between  groups  using  a  color 
coding  scheme  and  those  using  en  alphabetic  scheme  supports  previous 
findings  that  there  are  'no  clear  and  consistent  advantages  for  any  one 
visual  code  set  over  the  other*'  L0HR1 83 ,  p.  83).  Both  color  and 
alphabetic  schemes  are  excellent  for  locating  objects  CDAVI83).  TRAP 
event  longevity  probably  also  contributes  to  comparable  performance  with 
either  single-featured  coding  scheme. 

0.1. 1,3  Summary  of  Primary  Result.  The  major  resaarch  outcome  suggests 
that  efforts  in  both  Information  systems  researoh  and  decision  support 
system  design  could  benefit  greatly  from  a  focus  on  aiding  the  decision 
process  itself.  Decision  support  technology,  particularly  alternative 
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computer-based  form*  of  presenting  information,  have  been  th«  focus  of 
many  research  of  fort*,  Advance*  in  graphic*  technology  makt  research  in 
applying  th*  new  tool*  tantalising,  Th*  r«*ult*  of  thi*  itudy  <ugg«*t, 
howaver,  that  r«**aroh*r*  hav*  btan  probing  only  th*  tip  of  th*  io*b*rg. 
A  greater  gain  in  und*r*tanding  d*oi*ion  making  and  in  learning  how  to 
improve  d*oi*ion  quality  depend  on  probing  what  lie*  beneath  th*  surface 
--  the  process  by  which  decision  are  made. 

This  study  illustrates  the  benefits  of  applying  simple  deolsion 
rules  to  a  dynamic  group  intellective  task  involving  uncertainty,  Teams 
with  heuristics  (or  using  heuristics-oriented  strategies)  adopted  a  set 
of  behaviors  that  contributed  to  improved  deolsion  quality.  The 
behaviors  they  adopted  represent  departures  from  the  biases  evident 
among  teams  without  th*  heuristics.  Since  the  deolsion  processes  used 
by  the  groups  appear  to  have  been  related  to  group  member  biases, 
Huber's  program  model  seems  to  provide  a  reasonable  description  of  the 
group  decision  process  in  the  context  of  this  study  (HUBE81J.  Further, 
it  seems  to  be  possible  to  directly  intervene  in  the  group's  deolsion 
process  and  to  overcome  member  biases  by  providing  a  suggested  decision 

e 

strategy  (training), 

This  study  should  be  considered  as  an  early  exploratory  study  of 
the  effects  of  deolsion  process  interventions  on  group  performance,  Just 
as  th*  need  for  programs  of  research  (rather  than  one-shot  experiments) 
was  stressed  for  clarifying  th*  impact  of  graphics  on  decision-making 
[ DICK80 ,  BENBSOc),  progress  toward  understanding  th*  impact  of  decision 
heuristics  will  be  best  achieved  through  programs  of  research, 
Additional  studio*  on  the  impact  of  different  types  of  heuristics  on 
tasks  with  different  levels  of  structure  are  needed  to  fully  explain  the 
relationships  of  heuristics-aided  decision  processes  to  decision  quality 
and  group  perceptions. 

0.1.2  Discussion  of  Other  Findings. 

The  discussion  for  eaoh  of  th*  following  findings  explains  th* 
results  in  the  context  of  relevant  theory  and  related  research  findings 
and  identifies  implications  of  th*  findings  for  research  and  practice. 
0.1. 2.1  Lack  of  Findings  Related  to  Perceived  Workload,  SWAT  ratings 
were  captured  as  an  alternative  and  potentially  more  sensitive  measure 
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of  the  impact  of  the  independent  variables  in  the  TRAP,  but  SWAT  ratings 
were  not,  in  fact,  more  sensitive  than  TRAP  scores,  SWAT  ratings 
captured  the  largest  effects  (e.g.,  the  effects  of  time  stress),  but 
missed  many  of  the  more  interesting  effects. 

Observational  findings  suggest  that  SWAT  ratings  may  not  always 
have  been  a  valid  measure  of  an  individual's  perception  of  the  workload 
imposed  by  a  particular  trial.  For  example,  when  faoed  with  a  question 
about  the  time  stress  imposed  by  a  just-completed  trial,  some 
participants  were  heard  to  ask  others  whether  the  trial  had  been  a  fast 
or  a  slow  one.  Some  participants  may  have  applied  an  algorithm  to 
routinely  assign  a  particular  rating  based  on  the  objective  speed  of  the 
trial  rather  than  on  their  perceptions  of  the  stress  involved.  Ratings 
of  the  psychological  stress  (confusion,  frustration,  etc.)  imposed  by  a 
trial  may  also  not  have  been  consistently  valid;  ratings  may  have  been 
influenced  by  the  level  of  performance  achieved  on  a  trial,  Team 
conversations  subsequent  to  low-sooring  trials  sometimes  indicated 
intent  to  increase  ratings  of  psyohologioal  stress,  even  though 
breakdowns  in  team  communication  or  coordination  rather  than  increased 
trial  difficulty  accounted  for  the  low  score.  Other  measures  of 
workload,  as  well  as  measures  of  deolsion  confidence,  satisfaction  with 
the  decision  process,  and  other  perceptions,  nhould  be  captured  in 
further  studies  to  more  fully  explain  the  impact  of  changing  task 
conditions . 

,  Given  the  longevity  of  TRAP  events  on  the  display,  aspects  of 
performing  the  TRAP  other  than  locating  and  identifying  the  events  may 
have  had  greater  influence  on  workload  perceptions,  Tor  example, 
assessing  the  value  of  committing  resources  to  various  possible 
combinations  of  events  is  a  frequently  recurring  and  a  potentially 
difficult  mental  task  in  the  TRAP  that  would  nob  be  influenced  by 
differences  in  coding  schemes,  Any  difference  in  the  event 
identification  workload  associated  with  a  particular  coding  scheme  may 
have  been  overwhelmed  by  the  demands  of  the  more  difficult  value 
assessment  task,  which  was  the  same  for  all  coding  schemes,  Knowing 
whiah  aspects  of  a  task  contribute  most  to  its  difficulty  should  aid  in 
determining  where  decision  aids  would  be  most  useful, 


The  lack  of  of  foot  of  th«  heuriatlea,  in  particular,  on  perceived 
workload  may  havt  batn  due  in  part  to  workload  aaaoaiatad  with  applying 
the  heuriatlea  to  a  daciaion  iituation.  Any  contribution  of  tha 
hauriatioa  themaelvea  to  tha  load  on  tha  human  information  proceaalng 
ayatem  could  countaraot  tha  binefita  of  routiniaing  TRAP  daoiaiona.  Tha 
nat  affact  would  than  involva  no  apparant  influanoa  of  tha  haurlatica  on 
perceived  workload.  Further  atudiaa  ahould  aanaaa  tha  impaot  of  varying 
tha  acopa  of  tha  auggaatad  daciaion  atratagy. 

6. 1.2.2  Effaota  of  Tima  Btraai.  Qroupa  parformad  battar  and  paroaivad 
lower  lavala  of  workload  undar  modarata  tima  praaaura  than  undar  high 
tima  praaaura.  Maaauraa  of  tha  af facta  of  tima  praaaura  in  thia  atudy 
rainforoa  aimilar  af facta  in  aarliar  atudiaa  [BR0W8B,  WILS87],  Tima 
praaaura  did  maka  tha  TRAP  mora  difficult  and  may,  tharafora,  ba  an 
appropriata  maohanlam  for  manipulating  taak  difficulty  in  othar  daoiaion 
anvironmanta ,  particularly  for  dynamic  intallactiva  tadka. 

Thia  atudy,  howavar,  axaminad  affaota  at  only  two  lavala  of  tima 
atraaa.  Eapacially  givan  tha  finding  that  tima  praaaura  intaractad  with 
tha  haurlatica  auch  that  tha  haurlatica  aaamad  to  loaa  thair 
af (ecti vanaaa  undar  high  tima  praaaura,  atudiaa  ara  naadad  to  examine 
affacta  at  othar  lavala  of  tima  praaaura.  It  would  ba  valuable,  for 
axampla,  to  identify  a  point  at  whioh  awltchlng  from  prooaaa  aupport  via 
haurlatica  to  expert  ayatam  aupport  might  ba  appropriata.  Additional 
atudiaa  ahould  aleo  aaaoaa  tha  Impaot  of  atraaa  aaaooiated  with 
variablaa  other  than  tima  praaaura  ( a . g . ,  fatigue,  advaraa  working 
conditiona,  limita  on  communication). 

8.1. 2. 3  Effacta  of  Practice.  Qroupa  parformad  battar  and  paroaivad 
lowar  lavala  of  workload  with  practioa.  It  ahould  ba  notad  that 
learning  aaaooiated  with  tha  dteopoat  part  of  tha  learning  curve  (tha 
Tower  Law  of  Practice  I0ARD831)  may  not  have  bean  reflected  In  tha 
reaulta  of  thia  atudy.  Taama  completed  a  training  ataalon  involving 
briefingi,  competency  taata,  demonatrationa ,  and  practice  triala  prior 
to  tha  atart  of  aoacion  1.  Learning  apparently  continued,  however,  into 
tha  experimental  aaaaiona. 

Practice  had  a  poaitiva  influanoa  on  daciaion  quality  in  thia 
atudy,  but  tha  nature  of  ita  influence  varied  for  different  bahaviora 


aaaociated  with  deciaion  quality,  Overall  performance ,  meaaured  a«  tha 
ratio  of  tha  taam'e  aaora  to  tha  oomputar  modal'a  aoora  on  tha  aama 
trial,  reached  ita  paak  in  aaaaion  3  and  than  maintainad  that  laval 
through  aaaaion  4,  generally  following  expectation!  baaad  on  tha  Power 
Law  of  Praotioa  t C ARD83 ]  and  rainforolng  tha  raaulta  of  aarliar  atudlaa 
t  BROW03 ,  WILS07 } ,  Howavar,  aooraa  for  bahavioral  factora  that 
contributa  to  ovarall  TRAP  performance  followad  diffarant  ourvaa,  Tha 
ratio  of  a  taam'a  point!  aarnad  from  oartain-valuad  avanta  paakad  in 
aaaaion  2,  than  raturnad  to  a  laval  aomparabla  to  that  in  aaaaion  1.  On 
tha  othar  hand,  information  aaaking  aotivity  ralatad  to  unoartain  avanta 
didn't  paak  until  aaaaion  4.  Taking  thaaa  two  faotora  and  tha  vidaotapa 
review  raaulta  together,  tha  laarning  prooaaa  could  ba  oharaotarlaad  aa 
following  two  phaaaa,  Xn  tha  drat  phaaa,  taama  tand  to  bacoma  mora 
afflciant  at  handling  TRAP  daoialona  according  to  an  initial  atratagy, 
In  particular,  givan  a  biaa  toward  certain-valued  avanta,  taama  bacoma 
capabla  of  completing  mora  oartain-valuad  avanta  in  a  givan  period  of 
time,  In  tha  aacond  phaaa,  taama  tand  to  make  adaptation!  to  thair 
atratagiae,  uaually  involving  divarting  aoma  attention  from  oartain- 
valuad  to  uncertain  avanta. 

Workload  perception!  ware  lowaat  at  tha  aama  point  that  daoiaion 
quality  reached  ita  paak  (aaaaion  3),  reinforcing  tha  link  to  tha  Powar 
Law  of  Praotica  (CARD83) ,  Howavar,  in  aaaaion  4,  workload  perception! 
raturnad  to  tha  higher  laval  of  aeaaiona  1  and  2,  It  could  ba  that  tha 
beneficial  effect!  of  learning  on  perceived  workload  in  thia  atudy  were 
confounded  by  fatigue  affect!.  Seaalona  1  and  2  were  conducted  in  a 
aingle  2-hour  aaaaion  with  Juat  a  ahort  break  in  between,  Thera  waa  no 
aignifioant  difference  in  parcaivad  workload  between  tha  two  aeaaiona; 
fatigue  in  aaaaion  2  could  have  counteracted  the  laarning  affaota. 
Seaalona  3  and  4  ware  conductod  in  another  2-hour  aaaaion  on  a  aaparata 
day.  Parcaivad  workload  waa  at  it!  lowaat  in  aaaaion  3  when  laarning 
waa  nearly  complete  and  tha  participant!  were  fraah;  with  no  further 
learning  into  aaaaion  4,  fatigue  oould  account  for  tha  perceptlona  of 
incraaaad  workload,  Replicating  thia  atudy  with  different  achedullng 
approachaa  would  help  to  aaparata  tha  laarning  and  fatigue  affaota, 
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8.1.3  Other  Observational  Flndlnss 

As  indicated  in  Chapter  8,  Kimble,  MoNeeae,  and  Goodyear  examined 
the  effect*  of  emergent  leadership  on  TRAP  performance  in  a  study  run 
concurrently  with  this  experiment.  They  found  no  statistical  evidence, 
using  measures  of  the  relative  frequency  and  duration  of  talking  among 
team  members,  that  emergent  leadership  had  any  effect  on  group  deoislon 
quality  C KIMB87 ] .  Reviews  of  the  data  and  videotapes  for  the  top  three 
and  bottom  three  teams,  however,  reveal  some  interesting  patterns. 
During  the  experimental  sessions,  the  time  spent  in  verbal  communication 
by  the  top  three  teams  averaged  nearly  five  times  that  spent  by  the 
bottom  throe  teams.  What  verbal  communication  there  was  among  the 
bottom  teams,  none  of  which  had  the  heurlstios,  was  dominated  by  one 
member  of  each  team.  The  top  two  teams,  both  of  which  had  the 
heurlstios,  shared  verbal  communication  time  fairly  equally  among  two  or 
all  three  team  members.  Communications  on  the  third-place  team,  which 
did  not  have  the  heuristics,  was  dominated  by  one  team  member.  These 
results  reinforce  the  expectation  that  the  heuristics  would  improve  the 
likelihood  that  at  least  two  team  members  would  'see'  the  oorrect 
answers  in  the  TRAP,  an  intellective  task  for  which  ‘truth,  supported, 
wins'  (MCGR84 ] . 

The  behavior  of  some  teams  suggested  that  time  spent  in  verbal 
communication  was  not  an  adequate  measure  of  either  communication  or 
emergent  leadership,  Some  teams,  including  several  that  achieved  high 
performance  scores,  tended  to  communicate  only  by  exception.  They  may 
have  set  priorities  for  committing  resources  to  the  various  types  of 
events  during  the  training  session  or  during  breaks  (which  wore  not 
videotaped),  Then  during  the  experimental  sessions,  team  members  seemed 
to  simply  move  their  cursors  in  unison  to  chosen  evonts  without 
verbalizing  directions.  They  verbalised,  then,  only  when  a  team  member 
deviated  from  the  expected  pattern  or  if  a  particularly  noteworthy 
situation  developed,  Additional  research  concerning  the  impact  of 
information  technology  on  communication  and  leadership  is  needed.  For 
example,  group  members  in  this  Study  could  set  what  actions  other 
members  took  on  the  display;  what  effects  would  private  screens  or 
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public  screens  available  only  on  demand  have  on  performance, 
communication  pattern*,  and  group  leadership? 

One  possible  mechanism  of  communication  and  leadership  in  the  TRAP 
was  ‘leadership  by  cursor  movement.'  The  videotapes  were  not  reviewed 
at  a  level  of  detail  sufficient  to  deteot  if  a  team  member  consistently 
moved  his/her  cursor  to  a  display  row  ahead  of  other  team  members.  In  a 
visually  oriented  task,  however,  such  movement  could  have  provided  a  cue 
for  others  to  move,  and  thus  could  have  served  communication  and 
leadership  purposes.  Further  studies  could  tap  the  richness  of 
information  from  videotaped  sessions  to  clarify  the  roles  that 
communication  and  leadership  play  in  dynamic  group  choice  tasks  like  the 
TRAP. 


6 . 2  Conclusions 

The  following  two  sections  identify  research  implications  of  this 
study's  limitations  and  provide  a  final  summary  of  the  key  findings. 
8.2.1  Limitations  and  Their  Implications  for  Researchers 

Following  are  some  implications  for  future  researchers  based'  on  the 
limitations  of  this  study. 

1,  The  benefits  of  the  heuristics  may  have  been  achieved  at  the 
expense  of  taxing  human  capacity  to  process  Information,  The  scope  of 
the  heurlstias  and  the  means  for  presenting  them  may  be  Important 
variables  that  should  be  tested  in  replications  of  this  study. 

'  2,  There  were  indications  in  this  study  of  strategy  changes  for 
some  teams  operating  without  heuristics.  Further  research  about  the 

nature  and  timing  of  strategy  innovations  among  teams  with  and  without 
heuristics,  including  an  analysis  of  the  process  by  whiah  a  team  decides 
to  adapt  its  strategy,  would  help  to  clarify  how  providing  decision 
rules  affects  the  deoision  process. 

3.  The  practice  variable  in  this  Study  may  have  been  aonfounded 
with  fatigue  effects.  Replicating  the  study  with  alternative 
experimental  session  Scheduling  approaches  would  help  to  clarify  the 
relative  impact  of  practice  and  fatigue. 

4.  To  the  extent  that  speed  of  recognition  could  contribute  to 
performance  on  a  dynamic  visual  task,  attention  theory  was  a  useful 
basis  for  Selecting  coding  Schemes  to  test,  Because  of  the  longevity  of 


events  on  the  TRAP  display,  however,  this  study  was  not  an  appropriate 
test  of  feature  Integration  theory  itself  (a  theory  of  attention).  A 
separate  study  would  be  required  to  detect  whether  the  single-featured 
codes  used  aotually  differed  from  the  oonjunotive  scheme  in  speed  of 
initial  event  identif ioation/iooatlon  in  a  TRAP  display. 

5.  SWAT  ratings,  as  alternate  measures  of  the  impact  of 
independent  variables  in  this  study,  were  not  as  sensitive  as 
anticipated.  It  would  be  useful  to  test  SWAT  ratings  against  other 
measures  of  perceived  workload  for  the  TRAP  and  for  other  mental  tasks. 
It  would  also  be  useful  to  examine  effeots  of  heuristics  in  the  TRAP  on 
other  perceptions  (satisfaction  with  the  decision  process,  confidence  in 
decisions,  etc.). 

0.2.2  Summary  of  Key  Findings 

The  most  important  finding  of  this  ."Study  was  that,  in  a  dynamic 
group  choice  task,  heuristics  had  a  muoh  greater  positive  impact  on 
decision  quality  than  did  varying  the  coding  scheme  by  which  decision 
data  was  presented.  This  result,  taken  in  combination  with  the  findings 
of  Cats-Baril  and  Huber  [CATS87],  Strongly  suggests  focusing  additional 
development  and  research  attention  on  how  to  directly  aid  the  decision 
process  (e.g.,  by  providing  decision  rules).  It  could  be  that  the 
technology  by  which  decision  Information  is  presented  is  less  important 
than  how  the  technology  is  used. 

A  second  major  finding  reinforces  the  aoncept  that  training  by  way 
of  providing  decision  rules  helps  to  overcome  human  biases  in  handling 
uncertainty  [FISC82 ] .  Decision-making  groups  tend  to  choose  between 
certain  and  merely  probable  gains  according  to  prospeat  theory  rather 
than  expected  utility  theory  IKAHN7BJ.  In  this  Study,  groups  without 
heuristics  also  relied  on  a  variety  of  strategies  other  than  one  based 
on  expected  utility  to  choose  among  probabilistic  gains.  Qiven 
heuristics  that  encouraged  choosing  according  to  expected  utility, 
however,  groups  adopted  the  recommended  strategy  and  thereby  improved 
their  performance.  Further  research  should  help  to  identify  other 
decision  environments  in  which  training  for  handling  uncertainty  would 
be  beneficial. 


10B 


REFERENCES 


ArrLse 

ARKE80 

BELLB2 

BELL69 

BENB04 

BENB85 

BENB80a 

* 

BENBBGb 

BENB06O 

BROW85 


Applegate,  L.  Idaa  Management  in  Organisation  Planning. 
Unpubliahed  doctoral  diaaar  tation ,  Univeraity  of  Arisona, 
1880, 

Arkai,  H, ,  Dawea,  R,,  and  Chriatanaen,  C.  ‘Factor! 
Influencing  tha  Uaa  of  a  Daciaion  Rula  in  a  Probabi 1 iatio 
Taak,’  Organ! tational  Behavior  and  Human  Daoiaion  Proctaaaa. 
Voluma  37,  i960,  pp.  93-110. 

Ball,  D ,  'Regret  in  Daoiaion  Making  Undar  Uncertainty,* 
Operation!  Raaaaroh.  Voluma  30,  Numbar  5,  September-Octobar 
198V,  pp.  091-081 . 

Ball,  D.  ‘Diaappointment  in  Daciaion  Making  Undar 
Uncartainty,*  Operation!  Raaaaroh.  Voluma  33,  Numbar  1, 
January-Fabruary  I960,  pp.  1-27. 

Banbaaat,  1.  'An  Analyaia  of  Raaaaroh  Mathodologiaa , '  in  Tha 
Information  Syatama  Raaaaroh  Challenge,  F.  W.  McFarlan,  Ed., 
Harvard  Buainaaa  School  Raaaaroh  Colloquium,  Boaton,  Maaa: 
Harvard  Buainaaa  School  Praia,  1964, 

Banbaaat,  I.,  and  Daxtar,  A.  ‘An  Exparimantal  Evaluation  of 
Graphical  and  Oolor-Enhancad  Information  Freaantation ,  ‘ 
Managamant  Solanca,  Voluma  31,  Numbar  11,  Novambar  1985,  pp, 
1348-1304. 

Banbaaat,  I.,  and  Daxtar,  A.  ‘An  Invaatlgation  of  tha 
Ef factivanaaa  of  Color  and  Graphical  Information  Fraaantation 
Undar  Varying  Tima  Conatrainta , '  MIS  Quarterly.  Voluma  10, 
Numbar  1,  March  1980,  pp.  58-83. 

Banbaaat,  I.,  Daxtar,  A.,  and  Todd,  P.  'Tha  Influanca  of 
Color  and  Graphical  Information  Praaantatlon  in  a  Managerial 
Deciaion  Simulation,*  Human -Computer  Interaction .  Voluma  2, 
Numbar  1,  1980,  pp.  05-92. 

Banbaaat,  1,,  Daxtar,  A.,  and  Todd,  P.  ‘An  Exparimantal 
Program  Invaatigating  Color-Enhancad  and  Graphical  Information 
Praaantatlon;  An  Integration  oi  tha  Findinga,'  Communication! 
of  tha  ACM.  Voluma  29,  Number  11,  Novambar  1980,  pp.  1094- 
1105. 

Brown,  C.,  and  Leupp,  D.  ‘Team  Performance  with  Large  and 
Small  Screen  Diaplaya,'  Harry  G.  Armatrong  Aeroapace  Medical 
Reaearch  Laboratory  Technical  Report,  AAMRL-TR-85-033 ,  Wright- 
Pattaraon  Air  Force  Baaa,  Ohio,  June  1985. 


110 


CARD83 

CATS87 

CHRI73- 

s 

CHR 163 

CLAR63 

CLEV83 

DAVI83 

DESAB4 

DESA03 

DESA88 

D1CK77 

DICK80 

EGET34 


Card,  8.,  Moran,  T. ,  and  Newall,  A,  The  Psychology  of  Huroy. 
f!n mnuHr  Infraction,  Lawrenoe  Erlbaum  Associates,  Hillsdais, 

H. J.  i  1883. 

Cats-Har i 1 ,  W. ,  and  Huber,  G.  'Decision  Support  Systems  for 
Ill-Structured  Problems:  An  Empirical  Study,  ElSl&lSH 
Scitnots ■  Vo luma  18,  Number  3,  Summer  1887,  pp.  330-372. 

Chriat,  R.  ‘Raviaw  and  Analyila  of  Color  Coding  Researoh  for 
Visual  Displays,  *  Human  Faotorg,,  Voluma  17,  Number  8,  Daoambar 
1878,  pp.  842-370. 

Christ,  R.,  and  Oordo,  0,  ‘The  Effects  of  Extended  Fraotloe 
on  tha  Evaluation  of  Visual  Display  Codas,  Human  Faqtoyft, 
Voluma  28,  Number  1,  Fabruary  1883,  pp.  71-84. 

Clarkson,  Q.  and  Pounds,  W.  'Theory  and  Method  in  tha 
Exploration  of  Human  Dacidion  Bahavior,  jjiduf trial  Manaflamanj 
Raviaw,  Voluma  3,  Number  1,  Fall  1883,  pp.  17-27. 

Cleveland,  W.  Tha  Elamants  of  flrjtf'htnjj  1&1&*  «»d*worth 
Advanoad  Books  and  Software,  Montaray,  CA,  1883. 

Davis,  E. ,  and  Sweaty,  R.  'Human  Factors  Guidelines  in 


Computer  Graphics:  A  Casa  Btudy,'  jin 
Man-Machine  StMlft,  Voluma  18,  pp.  1 13“ 


ntamatlonal  Journal  of 
133. 


DaSanotls ,  G,  -Computer  Graphics  as  Deois i°»Aidg ! 
Directions  for  Research,*  BlgMUOP  Egltfl&iA.  Fall  1884,  pp. 
483-487. 

DaSanotls ,  G. ,  and  Gallupa,  B,  'Group  Decision  Support 

Systems:  A  Nsw  Frontier,*  DATA  BAM,  Winter  1883,  pp.  3-10. 

DeSanctis,  G,,  and  Gallupa,  R.  ’A  Foundation  for  Group 

Deoision  Support  Systems  Design,’  Managamsnt Jjojencs ,  \olums 
33,  Numbar  8,  1887. 

Dickson,  C.,  Sann,  J.  and  Charvany,  N. 

Managamant  Information  Systems:  The  Minnesota 

Mnnajamant  Science,  Volums  23,.  Numbar  8,  May  1877,  pp.  813 

;)23. 

Dickson ,  G.,  DeSanotis,  G.  and  McBride,  D.  'Understanding  tha 
Effectiveness  of  Oomputar  Graphics  »er  Decision  s^PP°rt‘  A 
Cumulative  Experimental  Approach,’  Communications — ol  to*  ftv.SS1 
Oanulty  1888,  pp.  40-47, 

EKeth  H. ,  Virsi,  R.,  and  Garbart,  H.  'Searching  for 
Conjunctively  Defined  Targets,'  dzm&k — S  f 

Psychology:  Human  Fimptlpn  ,.|.nd  Volum#  10 • 

Numbar  1,  1884,  pp.  32-38. 


FINK80  Fink,  S.  Crlala  Management:  Planning  for  tha  Inevitable. 

AMACOM,  American  Management  AaaooiationT  I860.  "* 

FISC82  Kiaohhoff,  B,  ‘Debiaaing*  in  Judgment  Under  Uncertainty; 

Haurlatlca  and  Blaaea.  Kahneman,  D.,  at,  *1.,  Edi.,  Cambridge: 
Cambridge  Univeraity  Praia,  1082. 

F1S1180  Fiahar,  B.  Small  Group  Daoialon  Making.  New  York:  McGraw- 

,  Hill  Book  Company,  1080. 

FULL78  Fullar,  J.  ‘Optimal  Solution!  Varaua  'Good'  Solution!:  An 
Analyaia  of  Hauriatio  Daolaion  Making,'  Omega .  Voluma  0, 
Number  6,  1078,  pp.  470-484. 

GALL85  Qallupa,  R.  'Tha  Impact  of  Talk  Difficulty  on  tha  Uaa  of  a 
Group  Daolaion  Support  Syatam,'  Unpubliahed  Doctoral 
Diaaartation,  Univaraity  of  Minnaaota,  1065 . 

GALL88  Gallupa,  R.  'Experimental  Raaaaroh  into  Group  Daolaion 

Support  Syatama:  Praotioal  laauaa  and  Problem!,'  Proceeding! 
of  tha  10th  Annual  Hawaii  Intarnatlonal  Oonftranca  on  Syitam 
Soianoaa.  1088.  pp.  5 16-523 . 

GHAN82  Ghani,  J.,  and  Luak,  E.  'Human  Information  Prooaaalng 

Raaaaroh:  Ita  MIS-Daalgn  Conaaquanoaa , *  Human  Syatama 

Management.  Voluma  3,  1082,  pp.  32-40. 

GRET78  Grathar,  D.  'Raoant  Payohologioal  Studiaa  of  Bahavior  Under 
Uncartainty , '  Amarloan  Eoonomlo  Review.  Voluma  88,  Number  2, 
May  1078,  pp.  70-74. 

HACK83  Haokman,  J.  and  Morria,  0.  'Group  Taika,  Group  Intaractlon 

Prooaaa,  and  Group  Parformanoa  Ef faotivanaaa, '  in  Small  Group! 
and  Social  Intaraotlon,  Voluma  1.  Blumbarg,  A.,  at.  al.,  Eda., 
New  York:  John  Wiley  &  Sona,  1083. 

HARR67  Harry  G.  Armatrong  Aaroapaoa  Madioal  Raaaaroh  Laboratory, 

SubJactlva  Workload  Aaaaaamant  Tachnloua  (SWAT):  A  Uaer'a 
Oulda  (Draft),  Wright-Pattaraon  Air  Foroa  Baaa,  Ohio,  April 
1087, 

H0US88  Houaal,  T.,  Sawy,  0.,  and  Donovan,  P.  'Information  Syatama 

for  Orlaia  Managamant:  Leaaona  from  Southarn  California 

Ediaon,'  MIS  Quartarly.  Voluma  10,  Humber  4,  Dacamber  1080, 
pp.  380-400. 

HUBE81  Hubar,  G.  'Tha  Nature  of  Organiaational  Daciaion  Making  and 
tha  Daaign  of  Daolaion  Support  Syatama,'  MIS  Quartarly,  Voluma 
5,  Numbar  2,  June  1081,  pp.  1-10. 


112 


HUBE02 


HUBE84A 

HUBE84b 

HUBE85 

I  SENS  1 

IVES80 

IVES82 

JARVB5 

JARVSOa 

JARVSOb 

J0HN85 

KAHN79 


Huber,  3.  'Group  Decision  Support*  Systama  a*  Aida  in  the  Ufla 
of  Structural  Group  Management  Teahniquaa,'  DSS-62 
Transactlone .  Second  International  Conference  on  Daemon 
Support  Systems,  June  1982,  pp.  96-108. 

Huber,  G.  ' Iaauea  in  the  Deaign  of  Group  Decieion  Support 
Systems,*  MIS  Quarterly.  Volume  8,  Humber  3,  September  1984, 
pp.  195-204. 

Huber,  G,  "The  Nature  and  Deaign  of  Post-lnduitrial 
Organisation*, '  Management  Science.  Volume  30,  Number  8,  1984, 
pp.  928-931. 

Huber,  V.  ‘Managerial  Application*  of  Judgmental  Biaaea  and 
Heuristic*, J  The  Organisational  .  Behavior  . Teaching  Review, 
Volume  10,  Number  4,  1985-1986,  pp.  1-20. 

Isanberg,  D.  ‘Some  Effect*  of  Time-Pressure  on  Vertical 
Structure  and  Decision-Making  Accuracy  in  Small  Group*,' 

Qr.ganlamj?.nal_  Bihay  1  or . and  Human  Per  for  mange, ,  Volume  27, 

1981,  pp.  119-134. 

Ives,  B . ,  Hamilton,  8.,  and  Davis,  G.  'A  Framework  for 
Research  in  Computer-Based  Management  Information  System*,' 
Management  Science ■  Volume  26,  Number  9,  September  1980,  pp, 
910-934. 

Ivea,  B.  'Graphical  User  Interfaces  for  Business  Information 
Systems,'  MIS  Quarterly.  Speolal  Issue  1962,  pp,  15-47. 

Jarvenpaa,  &.,  Dickson,  G.,  and  DeSanotis,  G,  'Methodological 
Issues  in  Experimental  IS  Researoh:  Experiences  and 
Recommendations,'  MIS  Quarterly.  Volume  9,  Number  2,  June 
1983,  pp.  141-156. 

Jarvenpaa,  S.  'The  Effects  of  Choice  Tasks  and  Graphical 
Information  Displays  on  Cognitive  Strategies  and  Decision 
Making  Performance,'  Unpublished  Dissertation,  University  of 
Minnesota,  1986. 

Jarvenpaa,  S.,  and  Dickson,  G.  'Graphics  and  Managerial 
Decision  Making;  Research  Based  Guidelines,'  Management 
Information  Systems  Research  Center  Working  Paper,  M1SRC-WP- 
87-06,  University  of  Minnesota,  November  1986. 

Johnson,  E . ,  and  Payne,  J,  'Effort  and  Accuracy  in  Ohoice,' 
Management  Science.  Volume  31,  Number  4,  April  1985,  pp.  395- 
414. 

Xahneman,  D.,  and  Tversky,  A.  'Prospect  Theory;  An  Analysis 
of  Decision  Under  Risk,*  Kconometrlca,  Volume  47,  Number  2, 
March  1979,  pp,  263-291. 


113 


KEENS  0 

KIMBS7 

\ 

KLE 1 82a 

KLE I S2b 

KLE 185 

KRAE84 

LEMA80 

LEWI  82 

L1TT80 

LOOM82 

LOOM0O 

MCGR02 

MCUR84 


Keen,  P.  ‘MIS  Reaearoh;  Reference  Diaciplinel  and  a 
Cumulative  Tradition,*  Proceeding!  of  tht  F i r » t  Intarnational 
Conference  on  Information  Bvatema.  1880,  pp.  8-18, 


Kimble,  0 . ,  MoHeeee,  M. ,  and  Goodyear,  C.  ‘Emergent 
Leaderehip  and  Team  Ef  footiveneaa  on  a  Team  Reaouroe 
Allocation  Teak,*  Unpubliahed  Manuaoript,  Armetrong 
Aeromedioal  Reaearoh  Laboratory,  1087. 

Klein,  Q,  ‘Heuriatio  Influence  on  Deoiaion  Prooeaaea,* 
Unpubliahed  Doctoral  Diaaertation ,  Texaa  Tech  Univeraity, 
1882, 


Kleinmuntc,  D.  ‘The  Relative  Performance  of  Deciaion 
Strategiea:  Heuriatica  and  Taak  Structure,1  Unpubliahed 
Doctoral  Diaaertation,  Univeraity  of  Chloago,  1082, 


Kleinmunta,  D.  'Cognitive  Heuriatica  and  Feedback  in  a 
Dynamic  Deciaion  Environment,'  Management  Science.  Volume  31, 
Number  8,  June  1888,  pp,  880-702, 


Kraemer,  K . ,  and  King,  J.  'Computer-Baaed  Syatema  for  Croup 
Deciaion  Support,*  Unpubliahed  Paper,  Univeraity  of 
California,  Irvine,  Auguat  1884. 


LeMay,  M.  and  Hird,  E,  'Operator  Work  Load:  When  la  Enough 
Enough?’  Communlcatlona  of  the  ACM.  Volume  28,  Number  7,  July 
1880,  pp,  038-042. 


Lewie,  L,  'Facilitator i  A  Microcomputer  Deciaion  Support 
Syatem  for  Small  Croupe. *  Unpubliahed  Doctoral  Diaaertation, 
Univoraity  of  Louievllle,  1882. 

Little,  J,  'Reaearoh  Opportunitlee  in  the  Deoiaion  and 
Managment  Sciencee,*  Management  Science.  Volume  32,  Number  1, 
January  1880,  pp.  1  13, 

Loomea,  0.,  and  Sugden,  R.  'Regret  Theory;  An  Alternative 
Theory  of  Rational  Choiee  Under  Uncertainty,'  The  Economic 
Journal ■  Volume  82,  December  1882,  pp.  809-824. 

Loomea,  Q. ,  and  Sugden,  R.  'Diaappointment  and  Dynamic 
Conaiatency  in  Choice  Under  Uncertainty,'  Review  of  Economlo 
Studlea ,  Volume  LI1I,  1880,  pp.  271-282. 

McQrath,  J.,  and  Kravita,  D.  'Croup  Reaearoh,'  Annual  Review 
of  Paychology.  Volume  33,  1882,  pp.  185-230. 

McGrath,  J.  Group! ;  Interaction  and  Performance.  Englewood 
Cliffa,  N.J.:  Prentice-Hall,  1884. 


114 


M1LL56  Millar,  Q,  ‘Tha  Magical  Kumbar  Savan,  Plua  or  Minua  Two: 

Soma  Limit*  on  Our  Oapaoity  for  Processing  Information,*  Tha 
Psychological  Review,  Voluma  83,  Numbar  2,  March  1958,  pp.  81- 
87. 

MINE84  Minar,  7.,  Jr.  'Group  varaua  Individual  Daoilion  Making:  An 
Invaatigation  of  Fwvformanoe  Maaauraa,  Daoilion  Strategies, 
and  Prooaii  Loiiaa/Qaini , '  Orlandaational  Bahavior  and  Human 
Parf ormanoa .  Voluma  33,  1884,  pp.  112-124. 

MUHS81  Muhl,  W. ,  and  Juatif,  R.  ‘Group  Choioai  in  a  Simulatad 

Managamant  Gama,’  Simulation  A  Oamai.  Voluma  12,  Numbar  4, 
Daoambar  1981,  pp.  4B1-46B. 

0LAN7B  Oiandar,  F.  'Saaroh  Bahavior  in  Non-Simultanaous  Choioa 
Situations:  Satisficing  or  Maximising?*  in  Utility, 

Probability,  and  Human  Decision  Making,  Wandt  and  Vlak,  Eds., 
D.  Raidal  Publishing  Company,  Holland,  197B,  pp.  297-320, 

PATTQ3  Pa 1 1 i pat  1 ,  K. ,  Klainman,  D. ,  and  Ephrath,  A,  'A  Dynamic 

Dacision  Modal  of  Human  Task  Salaotion  Parformanoa , *  IEEE 
Transactions  on  Systam*.  Man,  and  Qybarnatlcs,  Voluma  13, 
Numbar  2,  1983,  pp.  14B-188. 

FAYN78  Payna,  J . ,  Braunstain,  M,  and  Carroll,  J.  ‘Exploring 
Pradaoisional  Bahavior:  An  Altarnativa  Approach  to  Daoilion 
Research ,  ‘  Organisational  Bahavior  and  Human  Performance, 
Voluma  22,  1978,  pp,  17-44, 

RAPQ75  Rapoport,  A,  ‘Rasaaroh  Paradigms  for  Studying  Dynamic 

Dacision  Bahavior,*  in  Utility.  Probability,  and  Human 
Dacision  Making.  Wandt  and  Vlak,  Eds. ,  D7  Reidei  Publishing 
Company,  Holland,  197B,  pp.  349-369. 

REMU06  Ramus,  W,  ‘Graduate  Students  as  Surrogates  for  Managers  in 
Experiments  on  Business  Decision  Making,*  Journal  of  Business 
Rasaarch ,  Voluma  14,  19S6,  pp.  19-2B. 

5AQI8B  Sagi,  D . ,  and  Julesa,  B.  ‘Detection  versus  Discrimination  of 
Visual  Orientation,"  Perception.  Voluma  14,  1888,  pp,  619-628. 

SLOV77  Slovio,  P . ,  Fischhoff,  B.,  and  Lichtenstein,  S,  ‘Behavioral 

Dacision  Theory,*  Annual  Raviaw  of  Psychology,  Voluma  28, 

'  1977,  pp.  1-39. 

STEE8 1  Gtaab,  R. ,  and  Johnston,  S.  ‘A  Computer-Based  Interactive 

Systam  for  Group  Decisionmaking,*  IEEE  Transactions  on 
Systems,  Man,  and  Cybernetics ,  Voluma  SMC-11,  Numbar  8,  August 
1981,  pp.  544-562 . 

TREI80  Traisman,  A.,  and  Galada,  G.  ‘A  Feature-Integration  Theory  of 
Attention,*  CoSnltlve  Psychology,  Volume  12,  1980,  pp.  07-136. 


115 


TREI 82  Treisman,  A.  'Perceptual  Grouping  and  Attention  in  Visual 
Search  for  Featureii  and  for  Objects,*  Journal  of  Experimental 
Psycholosy;  Human  Perception  and  Performance.  Voluma  8, 
Number  2,  1982,  pp.  194-214. 

TULG80  Tulga,  M. ,  and  Sharidan,  T.  'Dynamla  Decisions  and  Work  Load 
in  Multi  talk  Supervisory  Control,*  IEEE  Transactions  on 
Syattma,  Man,  and  Cybarnatloa.  Voluma  SMC- 10,  Numbar  8,  May 
1980,  pp.  217-232. 

TUf<082  Turoff,  M,  'Computar  Support  for  Qroup  varaua  Individual 
Decisions,*  IEEE  Transactions  on  Communloatlona .  Voluma  COM- 
30,  Numbar  1,  January  1982,  pp.  82-90. 

TVERY4  Tversky,  A.,  and  Xahnaman,  D.  ‘Jurlgmant  Under  Uncertainty: 

Hauriatioi  and  Biases,*  in  Utility,  Probability,  and  Human 
Decision  MaklnS.  Wendt  and  Vlak,  Eda.,  Amarloan  Aaaoolatlon 
for  the  Advancamant  of  Balance,  1974,  pp,  141-102, 

VANWB6  Van  Watering,  P.  ‘An  Experimental  Investigation  of  the  User 
Interface  for  Computer  Operators,*  MIS  Workshop  Presentation, 
Univorsity  of  Minnesota,  October  3,  1986. 

WATJ87  Wataon,  8.  *A  Study  of  Group  Decision  Support  System  Use  In 
Three  and  Four-person  Groups  for  a  Preference  Allocation 
Decision.*  Unpublished  Doctoral  Dissertation,  University  of 
Minnesota,  1987. 

WI L807  Wilson,  D . ,  MoNeese,  M. ,  and  Brown,  C.  ’Utility  of  Shared 
Vareue  Isolated  Work  Setting  for  Dynamic  Team  Decision- 
Making.'  Harry  G.  Armstrong  Aerospace  Medloal  Research 
Laboratory  Technical  Report,  AAMRL-TR-B7-072 ,  Wright-Patterson 
AFB,  OH,  December  1987, 

WOHL84  Wohl,  J.,  Entin,  E . ,  Kleinman,  D.,  and  Pattlpati,  K.  ‘Human 
Decision  Processes  in  Military  Command  and  Control,'  in 
Advances  in  Man-Machine  Systems  Research:  A  Research  Annual, 
Volume  1,  Greenwioh,  CT:  JAI  Press  Inc.,  1084, 

ZIOU88  Zlgurs,  I.,  Poole,  M. ,  and  DeSanotis,  G.  ‘A  Study  of 
Influence  in  Computer-Mediated  Decision  Making , '  Management 
Information  Systems  Research  Center  Working  Paper,  MISRC-WP- 
88-07,  December  1987, 


116 


APPENDIX  A 

SUBJECTIVE  WORKLOAD  ASSESSMENT  TECHNIQUE  (SWAT) 


The  Subjective  Workload  Aeaeaament  Technique  (SWAT)  provide#  a 
in  *nre  of  manual  workload  aeeociated  with  taak  performance.  H  waa 
d«v..pid  on  the  bail*  of  additive  conjoint  meaaurement  methodology  for 
application  originally  to  problem#  related  to  aircraft  pilot  workload 
[RE1D811.  It  baa  ainoe  been  validated  on  a  variety  of  laboratory  taaka: 
probability  monitoring,  memory  update,  oontinuoua  recall,  apatial 
memory,  linguietic  prooeaeing,  mathematical  prooeaaing,  grammatical 
riavoning,  mate  learning,  and  other#  tREIDSSl, 

SWAT  meaaurea  workload  on  three  dlmenaiona:  time  load,  mental 

effort  load,  and  payohologioal  etroee  load,  Each  dlmenalon  ia  Judged  on 
a  three-point  aoale;  a  definition  of  each  dimenaion  and  ita  aaaooiated 
acale  follow: 

Time  Load  (refer#  to  the  fraction  of  the  total  time  that  the 
aubjeot  ia  buay) 

1.  Often  have  apare  time,  Interruption#  or  overlap  among 
activitie#  ooour  infrequently  or  not  at  all. 

2.  Occaaionally  have  apare  time,  Interruption#  or  overlap 
among  aotivitiea  ooour  frequently. 

3.  Almoat  never  have  apare  time.  Interruption#  or  overlap 

among  aotivitiea  are  very  irequent,  or  ooour  all  the 
time . 

Mental  Effort  Load  (an  index  of  the  amount  of  attention  or  mental 
effort  required  by  a  taak  regardleaa  of  the  number  of  taaka  to  be 
performed  or  any  time  limitation) 

1.  Very  little  oonecioue  mental  effort  or  concentration 
required.  Activity  ia  almoat  automatic,  requiring  little 
or  no  attention, 

2.  Moderate  oonecioue  mental  effort  or  concentration 
required,  Complexity  of  activity  ia  moderately  high  due 
to  uncertainty,  unpredictability,  or  unfamiliarity , 
Conaiderable  attention  required. 

3.  Exteneive  mental  effort  and  concentration  are  neceeeary. 
Very  complex  aotivlty  requiring  total  attention. 
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*  Piyehololloal  Strali  Load  (rafari  to  tha  contribution  to  total 

workload  of  any  condition*  that  produoa  anxiaty,  fruitration  and 

onfuiion  whila  performing  a  talk  or  talk*) 

1.  Littla  aonfuiion,  rilk,  fruitration.  or  anxiety  exiita 
and  oan  ba  aaaily  aooommodatad. 

3.  Modarati  straai  dua  to  confuaion,  fruitration,  or  anxlaty 
notioaably  add*  to  workload.  Significant  oompanaation  ia 
roquirad  to  maintain  adaquata  parformanca, 

3.  High  to  vary  intania  itraia  dua  to  confuaion, 

fruatration,  or  anxiaty.  High  to  extrema  datarmination 
and  aelf-eontrol  raquirad. 

Tha  SWAT  ia  a  two-atap  prooadura  involving  a  ioala  davalopmant 
phaaa  and  an  avant  aooring  phaaa.  In  tha  ioala  davalopmant  phaaa,  aaoh 
aubjaot  parformi  a  aort,  ranking  all  37  poaiibla  oomblnationi  of  tha 
lovala  of  aaoh  dimamion  from  lowaat  to  highaat  workload.  If  thara  ll 
•uffioiant  agroamant  among  tha  aubjiotl*  ranklngi.  a  iingla  group  ioala 
ia  davalopad  uiing  conjoint  analyiii  (group  icalaa  hava  baan  poaiibla  in 
all  tha  itudial  to  data),  If  than  il  inluffioiant  agroamant,  aubgroup 
or  tvan  individual  icalaa  may  ba  davalopad, 

Tha  avant  aooring  phaia  involval  oollactlng  Judgmanti  from  aaoh 
•  ubj act  on  aaoh  of  tha  workload  dlmanaloni  following  aomplation  of  an 
activity,  For  thil  ralaarch,  tha  ioala  for  aaoh  dimamion  appaarad  on 
aach  aubjact'i  CRT  at  tha  and  of  aaoh  TRAP  ►rial.  Tha  combination  of 
Judgmanti  waa  than  aiilgnad  a  ioala  valua  from  tha  ioala  davalopad  in 
tha  acala  davalopmant  phaia. 
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APPENDIX  B 

SAMPLE  TRAP  INSTRUCTIONS 
Team  Reiource  Allocation  Problem 
Alphabetio  Coding 

Before  you  taka  leatl,  pleaie  taka  a  oard  which  will  determine 
whether  you  will  ba  taam  mambar  A.  B,  or  C  lor  tha  itudy.  (Subjacttf 
draw  cards  and  ara  seated  at  appropriate  seats.)  Each  taam  mambar  hai 
him/her  own  diaplay  and  control  box  at  tha  workatation.  You  will 
communicate  with  your  teammatea  through  tha  headphonea  provided  at  tha 
workatation.  Tha  mierophonaa  on  tha  headaeta  ara  called  lip  mikea 
because  they  ara  intended  to  ba  kept  nearly  touohing  your  1 1 pa .  To  hold 
tha  background  noiia  to  a  minimum  and  to  hear  your  voicaa  clearly, 
plane  remember  to  kaap  tha  mike  in  thia  poaitlon,  (Damonatrata) 

GENERAL  INSTRUCTIONS 

Thia  experiment  la  concerned  with  how  taam  mambara  work  with  one 
another  to  acoompllah  taika,  You  and  your  teammatea  will  work  together 
to  decide  how  beat  to  alloaate  team  reaouroea  (your  work  time)  (or  the 
good  o 1  the  team  in  a  talk  which  Involve!  the  prooeaaing  o(  varioua 
targeta,  You  will  work  on  aome  target!  .ourself,  and  on  other  target! 
with  one  or  both  of  your  teammatea,  Tha  major  portion  of  the  diaplay 

will  have  11  rowa.  Target!,  repreaented  aa  circle!  labelled  A,  B,  C,  D, 
and  E  will  appear  randomly  in  each  of  thaae  11  rowa.  You  will  earn 
point!  for  the  taam  by  working  on  them*  target!  before  their  time  rune 
out  and  they  leave  the  aoreen.  Working  on  a  target  limply  mean! 

■electing  a  target  by  uaing  the  button!  on  your  control  box  to  move  a 
curler,  preaiing  the  abert  button,  and  waiting  a  few  aeoonda  for  the 
target  to  be  prooeiied.  Beoauae  more  target!  than  you  can  poiaibly  work 
on  will  appear  on  the  acreen,  the  particular  target!  you  chooae,  and 
their  point  value!  to  the  team,  will  be  quite  important.  Therefore,  it 
a  Decenary  for  you  to  learn  how  the  point  value!  of  target!  are 
.’uka.'mined .  Plealo  liiten  carefully. 

ALPHABETIC  CODING 

The  point  value  of  each  target  depend!  on  two  thing!;  the  number 
of  requlir  ,  worker!  and  the  letter  that  appear!  on  the  target,  Overall, 
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the  point  value  of  a  target  ia  proportional  to  tha  number  of  required 

workara.  Tha  average  value  of  all  tha  target*  1*  3  points  par  person. 

Targets  requiring  one.  two,  or  three  workers  are  therefor*  worth  an 

average  value  of  3,  6,  or  0  points,  respectively ,  for  the  team.  Whether 

a  particular  target  is  worth  this  average  value  of  3  points  per  person, 
or  somewhat  more  or  less,  depends  on  the  letter  that  appears  on  the 
target. 

Sinoe  the  letter  A  suggests  Importance  or  urgenoy,  you  will  see 
that  targets  ooded  toward  the  beginning  of  the  alphabet  are  worth  more 
points  on  the  average  than  targets  ooded  later  in  the  alphabet.  With 
this  notion  of  alphabetic  order  in  mind,  let's  examine  the  specific 
point  values  assigned  to  the  different  targets.  (Give  subjects  the 
point  values  table,) 

The  point  values  assigned  to  the  different  targets  are  shown  in  the 
table . 

A  targets  are  worth  B  points  per  person 

--  B  targets  are  worth  4  points  per  person 

0  targets  are  worth  3  points  per  person 

--  D  targets  are  worth  3  points  per  person 

--  E  targets  are  worth  1  point  per  person 

You  should  now  be  able  to  determine  the  point  valuo  of  each  target 
by  knowing  the  number  of  required  workers  and  the  letter  that  appears  on 
the  target. 

POINT  VALUE  TEST I NO 

Because  your  understanding  of  the  point  values  is  critical  to  this 
study,  I  am  going  to  have  you  complete  a  short  test  to  demonstrate  your 

knowledge  of  the  point  values  of  each  target,  Before  taking  the  test 

pleas*  examine  the  summary  table  of  the  point  values  and  (eel  free  to 
ask  questions  about  it,  (Pause)  Do  you  have  any  questions  before  you 
take  the  test? 

(Give  subjects  the  test,  If  any  questions  are  missed,  discuss  the 
question  with  the  subject  to  insure  his  understanding  and  then  give  him 
a  new  test.  Repeat  this  procedure  until  all  subjects  have  answered  all 
the  questions  correctly.) 

We  are  ready  to  continue.  (START  APPROPRIATE  DEMO) 


DEMONSTRATION 


Thia  la  a  damonatration  ol  tha  TRAP  taak.  Aa  you  can  ««a,  thara 
at 'o  11  rowa  on  which  targaba  appaar  at  random  and  mova  acroaa  bha  aoraan 
from  latb  bo  righb.  Tha  black  aquaraa  in  coiumna  A,  6,  and  C  lndioaba 
which  oparabora  ara  raquirad  bo  work  on  aaoh  bargab.  Tha  a caia  ab  bha 
top  repraaanba  30  bima  unita. 

Working  on  a  bargab  ii  vary  aimpla.  All  you  do  la  mova  your 
markar,  a  graan  aabariak.  bo  a  bargab  row  and  praaa  bha  abarb  bubbon  on 
your  raaponaa  box.  Work  aubomabically  bagina,  and  altar  a  ahorb  bima 
(10  bima  unita),  your  baam  will  raoaiva  bha  appropriaba  numbar  of  poinba 
(or  oomplabing  bha  bargab.  Thaaa  poinba  ara  aubomabically  addad  bo  bha 
nccumulabad  poinba  diaplay  (ahow) .  Whan  you  bagin  work  on  a  bargab.  a 
black  bar  will  appaar  in  bhab  bnrgab'a  row.  Tha  bar  rapraaanba  bha  10 
bima  uniba  raquirad  bo  oomplaba  prooaaaing  of  bha  bargab.  Tha  bargab 
will  mova  bhrough  bha  bar  aa  bha  prooaaaing  prooaada.  Whan  bha  bargab 
ntovaa  cub  ol  bhq  bar.  prooaaaing  of  bhab  bargab  haa  baan  complabad.  In 
ordar  bo  oomplaba  a  bargab  bafora  ib  laavaa  bha  aoraan.  you  will  hava  bo 
abarb  ib  bafora  ib  raaohaa  bha  blaok  daahad  lina  (whiia  bha  bargab  ia  in 
bha  opporbuniby  window), 

Work  on  aaoh  bargab  can  ba  dona  only  by  a  particular  taam  mambar  or 
combination  of  baam  mambara.  Aa  bha  oonbrol  box  bafora  you  indloabaa. 
you  ara  aithar  baam  mambar  A.  B,  or  0,  You  can  work  only  on  thoaa 
bargab a  which  hava  a  black  aquara  in  your  oolumn,  If  a  bargab  haa  mora 
than  ona  black  aquara  in  front  of  ib.  both  or  all  bhraa  corraapondlng 
team  mambara  will  hava  bo  work  on  bha  bargab  at  tha  aama  bima  in  ordar 
bo  oomplaba  lb. 

To  work  on  a  bargab.  you  muab  mova  your  markar  bo  bha  corraapondlng 
black  aquara.  You  mova  bha  markar  by  praaaing  bha  bubboni  labalad  up 
and  down  on  your  oonbrol  box.  <3o  ahtad  and  mova  your  markar  around. 
Notice  that  if  you  praaa  bha  up  bubbon  whan  you  ara  on  bha  bop  row,  your 
marker  movaa  bo  bha  bottom  row,  Similarly,  if  you  praaa  tha  down  bubbon 
when  you  ara  on  tha  bottom  row,  bha  markar  movaa  bo  tha  top  row, 

Onca  you  hava  bha  markar  on  bha  row  corraapondlng  bo  bha  bargab  you 
wish  bo  work  on,  all  you  hava  bo  do  ia  praaa  bha  xtart  bubbon.  If  you 
«r«  bha  only  baam  mambar  raquirad  for  bhab  bargab,  work  automatically 
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begin*  «nd  the  black  square  will  turn  yellow.  However,  if  on*  or  more 
additional  t«am  members  arc  required  for  the  target)  the  black  square 
will  turn  pink,  This  means  that  you  are  waiting  to  work,  Work  will 
begin  only  when  all  the  required  workers  for  the  target  have  moved  their 
markers  to  the  target  and  pressed  their  start  buttons,  When  this 
occurs,  all  the  squares  will  turn  yellow  indicating  that  work  has  begun. 

While  you  are  working  on  a  target  or  waiting  for  another  team 
member  at  a  target  your  marker  will  turn  red.  You  oan  move  it  to  any 
row  you  choose  in  preparation  for  the  next  target  you  may  wish  to  start. 
When  you  become  free,  your  marker  will  return  to  its  green  color 
indicating  that  you  are  ready  to  press  the  start  button  for  another 
target, 

You  may  wish  to  stop  working  on  a  target  bofore  oompietlng  it.  To 
do  this  you  simply  press  the  RESET  button  on  your  oontrol  box,  Your 
marker  will  turn  green  indicating  that  you  are  free  to  start  another 
target,  If  others  wars  working  on  the  target  with  you ,  they  will  also 
have  to  press  their  RESET  buttons  to  work  on  a  different  target.  You 
will  receive  no  points  for  targets  which  are  not  fully  computed.  If 
you  choose  you  may  begin  to  process  the  target  over  again,  but  it  will 
take  a  full  10  time  units  to  complete  it.  The  RESET  button  is  also  used 
when  you  no  longer  wish  to  wait  for  other  team  members  at  a  particular 
target . 

Processing  of  each  target  takes  10  time  units  (TUs) .  A  TU  is  some 
arbitrary  number  of  seconds.  The  current  example  trial  has  a  TU  of  3 
(seconds.  It  takes  30  TUs  (in  this  example  90  seconds)  for  a  target  to 
move  completely  across  the  screen.  During  the  actual  experiment,  the 
number  of  seconds  for  a  TU  will  be  less.  That  is,  the  targets  will  move 
across  the  screen  more  quickly  and  the  time  spent  processing  each  target 
will  be  less. 

The  table  in  the  lower  right  hand  portion  of  the  display  indicates' 
whether  each  team  member  is  free,  waiting,  or  working.  A  black  square 
indicates  that  a  particular  team  member  is  free,  while  a  blinking  pink 
square  indicates  that  a  particular  team  member  is  waiting.  When  a 
particular  team  member  Is  working,  a  numeric  countdown,  in  TUs,  will 
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indicate  how  much  processing  timi  rsmains  unb.ll  the  team  member  will  be 
finished  with  the  current  target. 

The  countdown  for  eaah  target  will  start  at  30  TUi  when  a  target 
is  nt  the  left-most  part  of  the  ecreen,  and  deoreaee  at  a  constant  rate 
the  target  moves  to  the  right,  When  a  target  is  at  the  end  of  the 
opportunity  window  (the  black  dashed  line),  the  countdown  will  be  at  10 
TUs.  The  target  leaves  the  screen  at  0  TU.  Sinoe  each  and  every  target 
requires  10  TUi  for  processing,  knowing  how  many  TUs  a  target  will 
remain  on  the  screen  can  be  useful  to  you  as  you  deoide  which  targets  to 
work  on,  and  when  to  work  on  them.  In  addition,  comparing  this 
information  to  the  countdown  of  toam  members  who  are  currently  working 
(show),  can  provide  vital  information  about  whether  there  will  be  enough 
time  to  process  particular  targets,  For  example,  if  a  team  member  has  0 
TUs  remaining  before  completing  a  particular  target,  he  will  not  be  able 
to  complete  both  that  target  and  another  target  that  currently  has  only 
15  TUs  remaining. 

The  object  of  this  exercise  is  to  accumulate  as  many  points  as 
possible  as  a  team.  This  means  discussing  alternatives  with  the  other 
members  of  the  team  in  order  to  make  optimum  selection  of  targets.  As 
there  will  be  more  targets  than  the  team  oan  possibly  process, 
combinations  of  targets  should  be  selected  which  optimize  teem 
performance  and  total  point  count. 


TABLE  B.l  TRAP  Talk  Poirt  Values 


Targe  t 

E 

D 

C 

B 

A 

Point  Value 
(Per  Person) 

1 

2 

3 

4 

5 
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APPENDIX  C 

INSTRUCTIONS  FOR  TRAP  WITH  UNCERTAIN  EVENTS 

Now  that  you  art  familiar  with  the  baiio  TRAP,  we  would  lika  to 
introduce  a  variation.  In  ordar  to  study  how  taama  parform  with 
lncomplaba  or  unoartain  information,  we  hava  addad  avanta  for  which  tha 
point  vaiua  ia  initially  unknown,  Tha  taam  may  chooaa  bo  query  tha 
ayabam  to  gain  morn  information  about  thaaa  avanta  bafora  deciding 
whether  to  commit  to  procadiing  thnm. 

Tha  UNCERTAIN  EVENTS,  which  appaar  on  tha  aoraan  oa  black 
rectangles ,  all  require  three  persona  to  procaaa  them.  Thera  are  two 
features  of  these  avanta  which  may  vary: 

1.  Probability  of  Payoff:  the  probability  that  a  taam  will 

actually  gat  tha  points  for  processing  tha  unoartain  event  ia  either  SOX 
(high)  or  20X  (low) . 

2.  Point  Value:  there  are  two  possible  point  values  for  each 

level  of  probability  of  payoff. 

These  two  features  taken  together  result  in  four  types  of  uncertain 
events  as  shown  in  the  following  table: 

TABLE  0.1  Types  of  Uncertain  Events 
Probability 

of  Payoff  Point  Values 


High  (SOX) 

7  (21) 

3  (0) 

Low  (20X) 

2B  (64) 

12  (39) 

Events  with  a  high  probibility  of  payoff,  then,  have  an  BOX  probability 
of  giving  points  and  may  be  worth  either  7  points  per  person  (21  points 
for  the  team)  or  3  points  per  person  (9  points  for  the  team).  Events 
with  a  low  probability  of  payoff  have  a  20X  probability  of  giving  points 
and  may  be  worth  either  2B  points  per  person  (64  points  for  the  team)  or 
12  points  per  person  (39  points  for  the  team).  Each  of  the  four  types 
of  uncertain  events  is  equally  likely  to  occur. 
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Thar*  id  a  cost  in  lima  (2  TUa)  raquirad  to  gat  information  about 
the  event*.  Taking  into  account  tha  tima  coat,  tha  uncartain  avanta  ara 
on  the  average  about  equivalent  to  a  10-point  three-perdon  event  (3.3 
point!  par  pardon) . 

Initially  tha  uncartain  avantd  appear  on  the  acreen  ad  black 
rectoriglaa  with  no  information  about  tha  probability  of  payoff  or  point 
vAlue.  In  order  to  query  for  information,  all  three  members  must  move 
their  markers  to  that  row  and  press  their  start  buttons,  Just  as  you 
would  to  work  on  an  event,  Two  Ttls  after  the  three  team  members  have 
preeaed  their  atart  buttons,  information  about  the  event  will  appear  on 
the  black  rectangle,  The  information  will  include  either  an  (I  or  L,  for 
high  or  low  probability  of  payoff,  respectively,  and  the  total  number  of 
points  the  team  may  earn  for  processing  the  event. 

At  the  time  the  information  is  obtained,  the  team  may  choose  to 
process  that  event,  which  they  can  initiate  by  pressing  their  start 
buttons  a  second  time  (uncertain  events,  like  the  other  TRAP  events, 
require  10  TUs  for  processing) ,  or  they  may  choose  not  to  process  that 
event,  in  which  case  they  are  free  to  move  their  markers  to  another 
event  or  events,  The  team  may  choose,  if  time  allows,  to  query  for 
information  on  more  than  one  uncertain  event  before  selecting  an  event 
or  events  to  process. 

The  ruled  and  procedures  for  processing  the  uncertain  events  are 
exactly  the  same  as  those  for  the  standard  TRAP  events, 

Aa  with  the  basic  TRAP,  the  object  of  the  task  id  to  accumulate  ad 
many  points  as  possible  ad  a  team.  This  means  discussing  alternatives 
with  the  other  members  of  the  team  in  order  to  make  optimum  selection  of 
events.  The  uncertain  events  may  be  viewed  dimply  as  additional 
alternatives.  As  there  will  be  more  events  than  the  team  can  possibly 
process,  combinations  of  evsnts  Should  bs  selected  which  optimise  team 
performance  (total  point  count). 
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APPENDIX  D 

INSTRUCTIONS  FOR  TRAP  WITH  HEURISTICS 

A  heuristic  i«  a  ‘rult  of  thumb'  or  a  simplified  approach  to  a 
problem  that  grows  out  of  experience  with  similar  fituationi.  It  may 
not  always  load  to  the  bait  solution  to  a  given  problem,  but  it 
generally  provides  a  Mood  Solution  that  oan  be  identified  quickly  and 
easily,  For  example,  an  appropriate  heuristic  for  the  TRAP  task  might 
be  ,'Always  work  3-person  A  targets.'  People  who  have  had  experience 
with  TRAP  have  suggested  several  heuristios  for  TRAP,  and  we  enaourage 
you  to  discuss  and  to  use  the  heuristios  as  you  Start  to  perform  the 
task . 

HEURISTIC  *1:  In  addition  to  3-person  A  targets,  look  for  a  2-person  B 
target,  especially  with  a  1-perfon  A  target, 

A  2-person  B  target  worked  at  the  same  time  as  a  1 -person  A 
target  gets  13  points  for  the  team.  Of  the  'for  sure' 
targets,  only  a  3-person  A  target  or  three  1-person  A  targets 
worked  at  the  same  time  get  more  points,  Even  a  1-person  0 
target  worked  at  the  same  time  as  a  2-person  B  target  gets 
more  points  (11)  than  many  combinations  that  may  be  available, 

HEURISTIC  *2:  Next,  check  as  many  uncertain  targets  as  you  can, 

Immediately  take  LB4  or  H21;  ignore  L30  and  H9. 

Heuristic  *2  comes  from  the  expected  value  of  the  uncertain 
targets,  or  the  average  payoff  you'll  get  for  working  those 
targets  ever  time.  Taking  into  account  the  time  delay  to 
identify  the  targets,  the  uncertain  targets  are  worth  on 
average  about  10  points  for  the  team,  (In  other  words,  you 
should  take  combinations  of  'for  sure'  targets  that  are  worth 
more  than  10  points  before  you  check  uncertain  targets,)  You 
can  compute  the  expected  value  of  an  individual  uncertain 
target  by  multiplying  the  probability  of  payoff  by  the  points 
available.  For  example,  the  expected  value  of  an  L64,  one1' 
you  identify  it,  is  (0,2)  x  04  ■  10.0,  Therefore,  an  LB4  will 
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only  pay  off  on*  out  of  fiva  times,  but  the  payoff  ii  big  whan 
It  ootnaa.  Your  a cort  on  ona  trial  may  not  show  tha  banefit  of 
selecting  an  L84,  but  your  ovarall  average  score  will*  An  H21 
alao  gata  an  expected  valua  of  18.8,  ao  it  la  Juat  aa  valuabla 
*a  an  L84 .  Tha  axpaotad  valua  of  an  L36  or  an  HB,  though,  la 
only  7.2,  which  la  ao  low  that  thara  ara  almoat  alwaya  battar 
optiona  available. 

HEURISTIC  «3:  Ignora  D  and  S  targe ta. 

Thara  will  almoat  alwaya  ba  a  battar  combination  available 
than  ona  that  involvea  1-  and  3-  point-per-person  targata. 

HEURISTIC  14:  Keep  tha  team  Synchronised,  All  three  team  tnembori 

should  atart  a  target  or  targata  at  the  aama  time. 

If  you  aan't  start  working  a  target  vary  close  to  tha  aama 
time  aa  your  teammatea,  you're  battar  off  to  atay  tree  until 
they  ara  free  again  than  to  Start  working  a  1 -person  target 
that  comas  along  after  they've  started.  You'd  Just  and  up  in 
an  endless  round  of  waiting  for  someone  to  ba  free  to  Start 
team  targata. 

Do  you  have  any  questions  about  tha  heuristics?  (Answer  questions) 

You  may  now  have  8  minutes  to  discuss  tha  heuristics  and  any  other 
strategies  you  might  choose  to  adopt. 


HEURISTICS  HANDOUT 


HEURISTIC  *1:  In  addition  to  3-paraon  A  targata,  look  for  a  2-paraon  B 
targat,  aipacially  with  a  1-ptrion  A  targat. 

HEURISTIC  *2:  Naxt,  ol.jok  aa  many  unoartain  targata  aa  you  oan. 
Immadiataly  taka  L84  or  H21|  ignora  L36  and  H9. 

HEURISTIC  13:  Ignora  D  and  E  targata. 

HEURISTIC  *4:  V.aap  tha  taam  aynohroniaed .  All  thraa  taam  mambara 

should  atart  a  targat  or  targata  at  tha  aama  tima. 


« 
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;  C0N8ENT  FORM 

!i  ..  ..  .  . ■-«  having  -full  capacity  to  consent,  do 

hereby  volunteer  to  participate  in  a  reeeareh  study  entitled, 
"r>r<un  Resource  allocation  Problem,"  under  tha  direction  of  Ha, 
Denise  L.  Wllnon,  Dr.  Clifford  Brown,  and  MaJ  D.J.  MeBrido.  Tha 
due i al on  to  participate  in  thia  reeeareh  la  completely  voluntary 
on  my  part.  No  one  haa  eoaread  or  intimidated  me  into 
participating  in  thia  program.  X  am  participating  baeauaa  1  want 

to.  . . .  haa  adequately  anawered  any  and  all 

ouiMttlane  I  have  about  thia  study,  my  participation,  and 

procedures  involved.  1  underatand  that  i _ _ _ 

will  be  available  to  anawar  any  quaatlona  concerning  procedure* 
throughout  thia  atudy.  I  underatand  that  if  aigniflcant  new 
Hndlngn  develop  during  the  couree  of  thia  reaaarch  which  may 
relate  to  my  deeiaion  to  continue  participation ,  I  will  be 
Informed.  I  further  underatand  that  I  may  withdraw  thia  coneent 
at  any  time  and  diacontinua  furthar  participation  in  thia  atudy 
without  prejudice  to  my  entitlemrnta.  X  alao  understand  that  tha 
medical  monitor  of  thia  atudy  may  terminate  my  participation  in 
thia  atudy  if  he  or  aha  faaia  thia  to  ba  in  my  bast  Interest. 

I  understand  that  my  participation  in  thia  atudy  may  be 
photographed,  filmed  or  videotaped.  1  content  to  the  uoo  of 
thane  media  and  underatand  that  any  records  of  my  participation 
in  this  atudy  may  only  ba  diacloeed  according  to  federal  law, 
including  tha  Federal  Privacy  Act,  B  DSC  892a ,  and  lta 
implementing  regulationa. 

I  underatand  that  my  entitlement  to  medical  care  or  compensation 
in  the  event  of  injury  are  governed  by  federal  laws  and 
r«gulatiane,  and  if  I  desire  further  information  1  may  contact 


I  FULLY  UNDERSTAND  THAT  I  AN  MAKINB  A  DECISION  WHETHER  OR  NOT  TO 
PARTICIPATE.  I1Y  SIGNATURE  INDICATES  THAT  I  HAVE  DECIDED  TO 
PARTICIPATE  HAVING  READ  THE  INFORMATION  PROVIDED  ABOVE. 


Signature 


Date 


Time 


AN 

PM 


1  have  briefed  the  volunteer  and  anawered  questions  concerning 
the  reaeareh  project. 


.Signature 


Date 
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ADDENDUM  TO  THE  CONSENT  FOAM 
Experiment!  Team  Resource , Allocation  Problem 

You  are  invited  to  participate  in  an  experiment  cientgnad  to  study 
how  people  in  teams  work  with  ona  another  to  eompletm  *  number  of 
tasks.  Tha  situation  you  will  ba  exposed  to  haa  thaoratleal 
nimllarity  to  those  ancountarad  in  command,  control,  and 
communicatiena  <C*>  ayatama  of  tha  U8AP.  A  batter  under atandl ng 
of  the  proeanaaa  by  which  team  mambara  complete  taeke  will  anal  at 
in  improving  thaaa  C"  ayatama.  Your  exposure  to  tha  equipment  la 
limited  to  your  watehing  tha  CRT  acraan  at  a  dlatanea  of  about 
two  feet  for  approximately  two  houra  par  day  for  three  daye. 

Thl a  doea  not  Involve  any  known  risks.  L 

In  th«f  enperiment,  you  will  ba  obaarving  a  computer  generated 
dlaplay  of  a  rapraaantatl on  of  a  work  environment.  By  praaaing 
puahbuttona  on  a  raaponaa  box  you  will  work  on  taaka  individually 
and  with  your  team  mambara,  laeauae  there  will  ba  more  taaka 
available  to  you  than  you  ean  complete,  tha  particular  taaka  you 
arid  your  team  mambara  ehooaa,  and  whan  you  ehooae  them,  will  be 
of  primary  lntaraat.  You  will  re calve  further  detailed 
i natructiona  at  tha  beginning  of  tha  experiment. 

The  renponeea  you  make,  and  tha  timaa  at  which  you  make  them  will 
be  recorded  for  later  tnalyaia,  Audio  and  video  recordings  will 
alao  be  made  for  subsequent  atudy.  Your  name  will  ba  recorded 
along  with  tha  dates  and  timaa  at  which  tha  experiment  la 
performed.  Your  confidentiality  ae  a  participant  in  thia  project 
will  ba  protected.  Your  Identity  will  only  be  revealed  in 
accordance  with  tha  Privacy  Act,  8  USC  882  and  lte  implementing 
regulation*,  A  numeric  code  will  ba  uaed  to  Identify  the  data  in 
any  publication. 

Any  monetary  benefits  will  be  In  accordance  with  BRL/Alr  Force 
agreements. 

You  are  free  to  refuse  to  participate  or  to  withdraw  your 
participation  in  tha  experiment  at  any  time.  Doing  so  villi  not 
prejudice  your  relation  with  the  Laboratory  in  any  respect. 

Any  quentions  you  may  have  should  be  directed  to  Me.  Danina 
Wl 1  non  (57872)  or  MaJ  D.J.  McBride  (37870). 

Your  willingnens  to  participate  in  thin  enperiment  in  greatly 
appreciated.  Your  signature  indicates  that  you  have  decided  to 
participate,  having  read  the  information  provided  above. 

YOU  WILL  BE  GIVEN  A  COPY  OP  THIS  FORM  TO  KEEP. 


Signature 


Date 
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APPENDIX  F 

INSTRUCTIONS  FOR  SUBJECTIVE  WORKLOAD  ASSESSMENT  TECHNIQUE 

MtnUl  workload  refers  to  tho  amount  of  an  individual' •  capacity 
that  la  uiad  in  tha  parformano*  of  a  given  talk.  Tha  amount  of  oapaoity 
used  dapanda  on  tha  individual  (ability,  affort.  training,  experience, 
ate,)  and  on  tha  demand!  of  tha  system  and  tha  talk, 

Among  tha  mathoda  aval labia  for  maaiurlng  workload  ara  parformanea 
and  aubjaotiva  measure!,  Parformanea  measures  ara  baaad  on  tha 
assumption  that  parformanea  daolinai  ai  workload  approaohaa  tha  uppar 
limit  of  an  individual'!  oapaoity  to  axart  affort,  In  othar  word*,  whan 
workload  approaohaa  an  individual'!  oapaoity.  ha  or  lha  bagini  to  maka 
arrori,  Howavar.  ainoa  paopla  band  to  adapt  to  talk!  and  to  hold  thair 
parformanea  oonitanb  ovar  a  ranga  of  condition! .  parformanea  maaiural 
may  not  raflaot  adequately  tha  impact  of  ohangoi  in  tha  oonditionl. 
Balora  parformanea  braaki  down,  tha  individual  might  ba  working  hardar 
to  avoid  making  arrori.  Tha  individual1!  lubjaotiva  faalingi  oould  than 

ba  uaad  to  indioata  tha  additional  affort,  providing  a  mora  laniibiva 

% 

maaaura  of  tha  impaot  of  changing  talk  oonditionl. 

Tha  Subjaotiva  Workload  Aiaaiimant  Taohniqua  (SWAT)  haa  baan 
davalopad  by  tha  Air  Forea  Aarolpaoa  Madioal  Raiaareh  Laboratory  to 
•nawer  tho  naad  for  a  aubjaotiva  maaaura  of  workload.  SWAT  ia  a  two 
dbap  procaai  coniiiting  of  a  aoala  davalopmant  phada  and  an  avant 
scoring  phaaa.  During  tha  loala  davalopmant  phaaa,  participant!  provide 
tha  data  naoaiiary  to  develop  a  workload  loale  for  each  individual.  At 
th*  avant  aooring  phaia,  tha  individual!  rata  tha  workload  aiaociatod 
with  a  particular  talk. 

You  ara  about  to  participate  in  tha  loale  development  phaia  of 
SWAT ,  which  involval  lorting  a  daok  of  oardl.  SWAT  dlltinguilhaa  three 
dimension!  of  workload;  time  load,  mental  affort  load,  and 
psychological  itraii  load.  For  each  of  tha  three  dimensions .  three 
lev* Is  have  baan  defined.  Tha  three  levels  of  three  dimension!  yield  37 
possible  oombinationi.  To  help  ui  develop  your  workload  scale,  please 
rank  order  tha  37  oombinationi  aeeording  to  your  own  perception  of  tha 
workload  rapraiantad  by  each  combination,  Your  rank  ordering  will  than 
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be  converted  to  a  personalised  0-100  workload  scale  by  means  of  the 
mathematical  prootdupft  of  conjoint  analysis. 

In  completing  your  c«rd  sort,  pltftftft  oonaidor  tht  workload  imposed 
by  th«  combination  rapraftanttd  on  ftftch  card,  and  arrange  tht  card!  from 

4 

tht  lowatt  through  tht  hightit  workload  condition.  You  may  ult  any 
atrattgy  that  you  ohooct  in  rank  ordering  the  card!,  Ont  atrattgy  that 
hat  proven  useful  to  othartf  it  to  first  arranga  the  oards  into  a  number 
of  preliminary  stacks  representing  *  Hi gh • '  ‘Moderate,1  and  ‘Low1 
workload,  Individual  oards  can  be  exchanged  between  stacks,  if 
necessary,  and  than  rank  ordered  within  stacks,  Btaoks  can  then  be 
racombinad  and  ohteked  to  be  sure  that  they  represent  your  ranking  of 
lowaat,  to  highest  workload.  However,  the  choice  of  strategy  is  up  to 
you  and  you  should  choose  the  one  that  works  best  for  you, 

There  is  no  ‘school  solution*  or  one  correct  order  for  this 
problem,  The  correct  order  is  what,  in  your  judgment,  best  describes 
the  progression  of  workload  from  lowest  to  highest  in  general  rather 
than  for  any  specific  task.  That  judgment  differs  for  each  of  us.  It 
is,  important  that  we  are  not  asking  for  ,your  preference  on  workload, 
Some  people  may  prefer  to  be  ‘busy1  rather  than  ‘idle,1  but  we  do  net 
need  to  know  that  preference,  Instead,  we  need  information  on  how  the 
three  dimensions  and  the  three  levels  of  each  one  will  affect  the  level 
of  workload  as  you  see  it,  You  may  prefer  moderate  levela  over  low 
Invela  of  the  dimensions,  but  the  low  levels  should  impose  less 
workload. 

You  will  need  a  description  of  the  three  dimensions  of  workload  and 
definitions  of  the  levels  within  each  dimension  to  complete  your  oard 
sort,  I  will  provide  that  Information,  and  you  may  also  consult  your 
handout,  Flease  feel  free  to  ask  questions  at  any  time, 

TIME  LOAD 

Time  load  refers  to  the  fraction  of  the  total  time  that  you  are 
busy,  When  time  load  is  low,  sufficient  time  is  available  to  complete 
ell  of  your  mental  work  with  some  time  to  spare.  As  time  load 
increases,  Spare  time  drops  out,  and  soma  aspects  of  performance  over  lap 
end  interrupt  one  another,  This  overlap  and  interruption  oen  oome  from 
performing  more  than  one  task  or  from  different  aspects  of  performing 


tin  lame  talk.  At  higher  lwala  of  tlma  load,  aapooti  of  performance 
olt«n  occur  aimultaneoualy ,  v^u  are  oonitantly  buay,  and  intarruptlona 
are  vary  fraquant. 

Tima  load  may  ba  judged  on  a  ti.va*  point  acala: 

I .  Tima  Load 

1.  Of  tan  hava  apara  tlma,  Ir.te  rupUona  or  ovarlap  among 
activitiaa  occur  lnfraquantly  or  not  at  all. 

3.  Occaalonally  hava  apart  tlma.  Intarruptlona  or  ovarlap 
among  activitiaa  occur  frequently, 

3.  Almoat  navar  hava  apara  tima.  interruption'  or  ovarlap 
among  activitiaa  ara  vary  fraquant,  or  occur  all  tima. 

MENTAL  EFFORT  LOAD 

Tima  load  rafara  to  tha  amount  of  tlma  ona  haa  available  to  perform 
a  taak  or  taaka,  In  aontraat,  mental  effort  load  la  an  index  of  tha 
amount  of  attention  or  mental  effort  .required  by  a  talk  rtgardleia  of 
tha  number  of  taaka  to  ba  performed  or  ahy  time  limitation.  Vhrn  mental 
of  fort  load  ie  low,  tha  concentration  mid  attention  required  by  a  taak 

i  ■ 

ia  minimal  and  parformanoa  la  nearly  automatic.  Aa  tha  demand  for 
mental  effort  increaeea,  tha  degree  of  concentration  and  attention 
required  to  perforin  lnareaaea,  due  to  taak  oomplaxity  or  tha  amount  of 
information  which  mult  ba  dealt  with  in  order  to  perform  adequately, 
Ihgh  mental  effort  load  demand!  total  attention  or  concentration  due  to 
taak  oomplaxity  or  tha  amount  of  information  to  ba  dealt  with, 

Mantel  effort  load  may  be  judged  on  e  three-point  acala; 

II.  Mental  Effort  Load 

1.  Vary  little  conacloua  mental  effort  or  concentration 
required.  Activity  la  almoat  automatic,  raquiring  little  or  no 
attention. 

3,  Moderate  aonaoioua  mental  effort  or  concentration 
required,  Oomplaxity  of  activity  ia  moderately  high  due  to  uncertainty, 
unpredictability,  cr  unfamiliarity.  Oonaiderable  attention  required, 

3.  Sxtenaive  mental  effort  and  concentration  ara  neceiaary, 
Very  complex  activity  requiring  total  attention. 
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STRESS  LOAD 

Streaa  load  ptfara  to  the  contribution  to  total  workload  of  any 
condition!  that  prcduot  anxiety,  fruatration  and  oonfuaion  while 
performing  a  taak  or  taaka.  At  low  level!  of  payohologioal  atreaa,  one 
Uoir  relatively  relaxed,  Aa  atreaa  inoreaaea,  oonfuaion,  anxiety,  or 
fruatration  inereaae  and  greater  concentration  and  determination  are 
required  to  maintain  control  of  the  aituation. 

Streaa  may  be  judged  on  a  three-point  acale: 

III ,  Streaa  Load 

1.  Little  oonfuaion,  riak,  fruatration,  or  anxiety  exiata 
mid  can  be  eaaily  accommodated , 

2.  Moderate  atraa.i  due  to  oonfuaion,  fruatration,  or  anxiety 
noticeably  adda  to  workload,  Significant  compeneation  la  required  to 
maintain  adequate  performance, 

3.  High  to  very  intenae  atreaa  due  to  oonfuaion, 
fruatration,  or  anxiety.  High  to  extreme  determination  and  eel f -eontrol 
required , 

The  letter!  you  aee  on  the  baok  of  the  oarda  are  to  allow  ua  to 
arrange  the  oarde  in  a  prevloualy  randomiaed  eequenoe  ao  that  everyone 
geta  the  aame  order.  If  you  examine  your  deok  you  will  aee  the  order  on 
the  back  runa  from  A  through  Z  and  then  ZZ.  Theae  letter!  have  nothing 
to  do  with  your  arrangement  of  the  oardaj  they  aimply  allow  ua  to  put 
the  card!  back  into  the  aame  atarting  aequence  for  the  next  peraon  who 
aorta  the  deck. 

During  the  event  aooring  phaae,  event!  will  be  rated  uaing  the  lame 
dencr i ptora  uaed  for  acale  development,  Aaked  to  provide  a  8WAT  rating 
for  a  particular  event,  an  individual  would  aaaign  either  a  1,2,  or  3 
to  each  of  the  three  dlmenalona  of  time  load,  effort  load,  and  atreaa 
load  experienced  during  that  event,  The  number!  for  each  level  of  the 
three  dimenaiona  are  defined  aa  in  the  acale  development  phaae.  Theae 
three  rating!  correapond  to  one  of  the  combination!  created  in  the 
ordering  procedure  for  acale  development.  Your  workload  acale  value 
computed  for  thia  particular  combination  of  the  three  factor!  la  then 
the  aubjective  workload  acore  aaaignad  to  tha  avant, 


/• 
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From  thia  point  until  you  have  completed  the  iortin|  will  probably 
take  30  minutoa  to  an  hour.  Flaaaa  f •• 1  (rat  to  aak  quaationa  at  any 
time,  Thank  you  for  your  oooparation. 

SWAT  HANDOUT 
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I.  Tima  Load 

1.  Of  tan  hava  apara  tlma.  Intarruptiona  or  overlap  among 
activitiaa  occur  infrequently  or  not  at  all. 

2.  Ocoaaionally  hava  apara  tima.  Intarruptiona  or  ovarlap 
among  aotivitiaa  occur  frequently. 

3.  Aimoat  never  hava  apara  tima.  Intarruptiona  or  ovarlap 
among  activitiaa  are  vary  frequent,  or  ooour  all  the  tima. 

II.  Mental  Effort  Load 

1.  Vary  little  oonaoioua  mental  effort  or  concentration 
required,  Activity  la  aimoat  automatic,  requiring  little  or  no 
attention. 

2.  Moderate  oonaoioua  mental  effort  or  concentration 
required.  Complexity  of  activity  id  moderately  high  duo  to  uncertainty, 
unpredictability ,  or  unfamiilarity,  Gonalderable  attention  required. 

3.  Extonalvo  mental  effort  and  concentration  are  neoeeaery, 
Very  complex  activity  requiring  total  attention. 

III.  Strata  Load 

1.  Little  oonfuaion,  riak,  fruatration,  or  anxiety  axiata 
and  can  be  oaally  accommodated. 

2.  Moderate  etroaa  due  to  oonfuaion,  fruatration,  or  anxiety 
noticeably  addi  to  workload.  Significant  companaation  ia  required  to 
maintain  adequate  performance, 

3.  High  to  very  lntenae  atreaa  duo  to  oonfuaion, 
fruatration,  or  anxiety,  High  to  extreme  determination  and  eelf*control 
required. 
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APPENDIX  G 


Team  Resource  Allocution  Problem 
Test  of  Point  Value  Understanding 
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Name  ■ 

bate 

One-Person  Events 
1 .  If  A  processes 
points. 

0.  If  C  processes 
points. 

3.  If  C  processes 
points. 


_  Teem  ______  Worker  i  A  B 

Session  _ _  . 

i, 

the  team  will  accumulate  _ 

the  team  will  accumulate  . . 

the  team  will  accumulate  _ 


C 


Two-person  Events 

<1.  If  AB  together  process 

_____  points. 

t 

3.  if  BC  together  process 
_____  points. 

I  hr os -Per son  Events 

6.  if  ABC  together  process 
_  points, 

7.  If  ABC  together  process 
_ points. 

>J.  If  ABC  together  process 
_  points. 


the  team  will  accumulate 


the  team  will  accumulate 


the  team  will  accumulate 


the  team  will  accumulate 

the  team  will  accumulate 
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Team  Resource  Allocution  Problem 
Teat  of  Heuristics  Understanding 
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Name  _____________________________  Team  _ __ ___  Workeri  ABC 

Data  ___________ __________________  Session  _ 

Each  of  th«  following  diagrami  represents  a  limitad  version  of  a 
situation  you  could  face  in  tha  TRAP.  Using  tha  heuristics  you 
have  learned,  examine  each  diagram,  'select  the  action  that  is 
most  appropriate,  and  respond  to  the  question  following  the 
diagram. 


ABC 


■ 

"" 

r 

©  ! 

□ 

■  ©  I 

■ 

^**w» 

*-4- 

■M 

■■■ 

MHHiM  1 1  || 

Please  indicate  which  line  or  combination  of  lines  your  team 
should  work  on i  _ 


ABC 


□ 

:[ 

immggggggM^^^^m 

■ 

-! 

m 

BS 

■  ' 

ESS  d 

i 

jf[ 

‘IbhBBnh 

Please  indicate  which  line  or  combination  of  lines  your  team 
tdiuuld  work  oni  _ 
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Plan**  indicata  tha  baat  action  for  paraon  C  (placa  an  X  riant  to 
Ilia  baat  option)  I 


bat  taam  to  raaat  and  start  work  on  lira  1 
Buggaat  Una  2  aa  next  tar  gat  altar  eomplating  lina  S 
Buggait  lina  3  aa  naxt  targat  altar  eomplating  lina  3 
Start  wort:  on  lina  4 

Gat  taam  to  raaat  and  start  work  on  lina  6 


APPENDIX  I 
STATISTICAL)  RESULTS 

This  appendix  contains  the  detailed  reeulte  for  all  dependent 
variables  used  In  the  study.  Tables  1.1  through  1.9  provide  cell  means 
end  variances  lor  eaoh  ol  the  following  variables: 

.  -  SCORE  *  (TEAM  SCORE)  *  100  /  (MODEL  SCORE) 

--  XCERT  >  PROPORTION  CERTAIN 
-  1PL-1PT  ■  1  PLAYER,  1  POINT/PLAYER  EVENTS 

2FL-2PT  ■  2  PLAYERS,  2  POINTS/PLAYER  EVENTS 

--  2PL-4PT  ■  2  PLAYERS,  4  POINTS/PLAYER  EVENTS 

*  ■*  3PL-SPT  •  3  PLAYERS,  8  POINTS/PLAYER  EVENTS 

INFO  ■  INFORMATION  SEEKING  (INFO/SI  indicates  Information 
seeking  activity  in  experimental  session  1) 

*-  COMMIT  •  RESOURCE  COMMITMENTS  (proportion  of  uncertain  events 
to  which  resources  are  committed  subsequent  to  seoking 
information) 

--  SWAT 

The  tables  of  means  and  variance#  are  followed  by  Tables  I. 10  and  I. 11, 
which  present  raw  data  for  all  variables. 


TABLE  1.1  CELL  MEANS  AND  VARIANCES 
SCORE  ■  (TEAM  SCORE)  *  100  /  (MODEL  SCORE) 
(n  ■  4) 


TABLE  1.2  CELL  MEANS  AND  VARIANCES 
PROPORTION  CERTAIN 
(n  ■  4) 


With  Htur istioa 
Station 


Without  Hturiatica 
Station 


3 

2 

3 

4 

1 

2 

3 

4 

Color 

Faat 

mtan 

atd 

0.303 

0.000 

0.320 

0.072 

0.325 

0.002 

0.312 

0.003 

0.071 

0.158 

0.785 

0.225 

0.707 

0.227 

0.051 

0.123 

Mod 

mtan 

atd 

0.330 

0.074 

0.355 

0,050 

0.334 

0,110 

0.345 

0,039 

0.010 

0.188 

0.703 

0.217 

0.032 

0.253 

0.002 

0.200 

Alpha 

Faat 

mtan 

atd 

0.510 

0.144 

0.507 

0.105 

0.474 

0,120 

0.403 

0.101 

0.500 

0,202 

0.024 

0.105 

0.530 

0,313 

0.553 

0,248 

Mod 

mtan 

atd 

0.521 

0.060 

0.524 

0,150 

0.508 

0.075 

0.400 

0,043 

0.505 

0,145 

0.558 

0.190 

0.404 

0.214 

0.470 

0,252 

CoriJ 

Faat 

mtan 

atd 

0,343 

0.082 

0,411 

0.048 

0,487 

0,106 

0.438 

0,102 

0,005 

0,281 

0.500 

0,247 

0.548 

0.105 

0.405 

0,243 

Mod 

mtan 

atd 

0.372 

0.002 

0.444 

0.027 

0.410 

0.008 

0,504 

0.053 

0.512 

0.103 

0.578 

0.174 

0.518 

0.210 

0,408 

0.183 
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TABLE  I. 3  CELL  MEANS  AND  VARIANCES 
1  PLAYER,  1  POINT/PLAYER  EVENTS 
(n  *  4) 


With  Hauriatica 
Station 


Without  Hturiatioa 
Staaion 


1 

2 

3 

4 

l 

2 

3 

4 

Color 

Paat 

moan 

atd 

0,090 

0,023 

0.010 

0,020 

0.030 

0,020 

0.000 

0,000 

0.118 

0,099 

0,040 

0,048 

0.126 

0.173 

0,098 

0.073 

Mod 

moan 

atd 

0.030 

0,090 

0.030 

0,090 

0,020 

0,023 

0.010 

0,020 

0.118 

0,073 

0.170 

0,087 

0.128 

0,  102 

0.118 

0,073 

Alpha 

Faat 

tnaan 

atd 

0,090 

0,023 

0.040 

0.033 

0,083 

0,074 

0.040 

0.033 

0,040 

0.033 

0,080 

0.038 

0.030 

0.038 

0.083 

0.074 

Mod 

moan 

atd 

0,040 

0.033 

0.085 

0.075 

0,083 

0,081 

0.093 

0,058 

0.043 

0.078 

0.093 

0.087 

0.053 

0.058 

0.083 

0.084 

Con  j 

Faat 

mtan 

atd 

0.010 

0,020 

0.000 

0.000 

0.010 

0,020 

0.083 

0.113 

0.080 

0.023 

0.073 

0.120 

0.073 

0,099 

0.063 

0,100 

Mod  i 

mtan 

atd 

0.020 

0.040 

0.010 

0,020 

0.040 

0,048 

0.070 

0.020 

0.080 

0 . 038 

0.080 

0,020 

0.093 

0,087 

0.083 

0,081 

TABLE  1.4  CELL  MEANS  AND  VARIANCES 
2  PLAYERS,  2  POINTS/PLAYER  EVENTS 
(n  ■  4) 


With  Hturiatioa  Without  Hturiatlca 

Station  8e  ^ 1  on 


1 

2 

3 

4 

1 

2 

3 

4 

Color 

Faat 

main 

atd 

0.000 

0.000 

0.010 

0,020 

0,000 

0,000 

0.000 

0.000 

0,030 

0,038 

0,083 

0.001 

0.020 

0.040 

<',02f 

0,023 

Mod 

mtan 

atd 

0.010 

0,020 

0.000 

0,000 

0,000 

0,000 

umi 

0.033 

0,003 

0.000 

0.073 

0.008 

0.004 

0.073 

0,073 

Alpha 

Faat 

mtan 

atd 

0,010 

0.020 

0,000 

0,000 

0.010 

0.020 

0.000 

0.000 

0.003 

0.081 

0.010 

0,020 

0.010 

0.020 

0,043 

0,061 

Mod 

mtan 

atd 

0,030 

0.038 

0,010 

0,02* 

0.030 

0,020 

0.010 

0,020 

0.010 

0.020 

0.040 

0,040 

0.020 

0.040 

0.000 

0,000 

Conj 

Faat 

mtan 

atd 

0.000 

0.000 

0.010 

0,020 

0.020 

0,023 

0.030 

0,038 

0.003 

0.081 

0.003 

0,100 

0.033 

0.068 

0.063 

0,100 

Mod 

mtan 

atd 

0.073 

0.004 

0.020 

0.023 

0.020 

0.023 

0.020 

0.023 

0.003 

0.004 

0,000 

0.044 

0.008 

0.044 

0.003 

0.081 

I-B 


TABLE  1,9  CELL  MEANS  AND  VARIANCES 
2  PLAYERS,  4  POINTS/PLAYER  EVENTS 
<n  -  4) 


With  Hturiatica  Without  Hturiatica 

Station  Station 


1 

2 

3 

4 

1 

2 

3 

4 

Color 

Paat 

main 

ltd 

0,290 

0, 143 

0.228 

0,100 

0,213 

0,118 

0,108 

0,063 

0.833 

0,104 

0,888 

0,280 

0.823 

0.171 

0,308 

0,181 

Mod 

mttn 

atd 

0,303 

0.193 

0.369 

0.138 

0.290 

0,207 

0,389 

0,078 

0.623 

0.187 

0.963 

0.272 

0.833 

0.387 

0.480 

0,189 

Alpha 

Paat 

mttn 

atd 

0,368 

0,334 

0.463 

0,299 

0,460 

0,281 

0,309 

0,178  1 

0.283 

0,224 

0.343 

0,272 

0.280 

0.301 

0.323 

0,327 

Mod 

mttn 

atd 

0,873 

0,287 

0.893 

0,217 

0,820 

0,283 

0.843 

0.122 

0,348 

0,103 

0.470 

0,264 

0.313 

o.aoe 

0.318 

0.398 

ConJ 

Ptat 

mttn 

atd 

0, 168 

0,106 

0,308 

0.119 

0,323 

0.180 

0,338 

0,143 

0.460 

0,230 

0.468 

0,210 

0.440 

0.226 

0.346 

0.227 

Mod 

mttn 

atd 

0.263 

0,107 

0.488 

0.107 

0,388 

0.124 

0,823 

0,110 

0.470 

0,188 

0,483 

0,189 

0.428 

0.233 

0,408 

0,174 

K 


TABLE  I. fl  CELL  MEANS  AND  VARIANCES 
3  PLAYERS,  B  POINTS/PLAYER  EVENTS 
(n  ■  4) 


With  Hiuriitioi 
Stiiion 


Without  Hturiitioi 
Stiiion 


1 

3 

3 

4 

1 

3 

3 

4 

Color 

Fait 

mi  an 

ltd 

0,608 

0,119 

0.813 

0,339 

0.970 

0,060 

0.908 

0,119 

0.970 

0,060 

0.970 

0.060 

1.000 

0,000 

0,878 

0,103 

Mod 

moan 

ltd 

0,808 

0,118 

0,808 

0,116 

0,878 

0,380 

0,848 

0,130 

0,838 

0.138 

0,878 

0.380 

0.040 

0.080 

0.848 

0,188 

Alpha 

Fait 

mian 

ltd 

1,000 

0.000 

0,878 

0,174 

0,970 

0.060 

0.840 

0,069 

0,838 

0,138 

0.870 

0,080 

0.878 

0,380 

0,878 

0,174 

Mod 

maan 

ltd 

0,870 

0,080 

0,870 

0,060 

0,940 

0,069 

0.638 

0.138 

0.808 

0,119 

0,640 

0,089 

0.848 

0.337 

0,848 

0,310 

ConJ 

Fait 

moan 

ltd 

0,808 

0.119 

0.970 

0,080 

H 

0.848 

0,310 

0.876 

0,380 

0.818 

0,078 

0,818 

0,317 

Mod 

mi  an 

ltd 

0,908 

0.119 

1.000 

0,000 

1,000 

0,000 

1.000 

0,000 

0,840 

0,089 

0,808 

0,119 

0,840 

0.069 

0,970 

0,060 

v; 

i 


119 

mtan 

ltd 

H31 

mtan 

ltd 

L36  mean 
«td 


L84  mtan 
■  id 


Mod  H9  mean 

■  id 

H31  mtan 
aid 

L30  mtan 

■  id 

L84  mtan 


0,584 

0,151 


0.538 

0,138 


0,578 

0,159 


0.531 

0.143 


0.540 

0,118 


0.050 

0.101 


o.eee 

o.ioe 


0,013 

0.140 


-1  m 

k«w 


im'.i*  .i*  jt*  . 


•  at..  **-'*»..  i‘*.  k 


.7  CELL  MEANS  AND  VAR1AN0ES 
INFORMATION  SEEKING 


(n  -  1) 

ith  Hturiitiol  Without  KauriatlQl 

Station  Station 


3 

3 

4 

1 

3 

3 

4 

0,839 

0.483 

0.588 

0.313 

0.396 

0.338 

0.3881 

0,108 

0.111 

0.173 

0,140 

0,337 

0,160 

0,103 

0.500 

0.538 

0.858 

0.306 

0.331 

0.388 

0.396 

0.18b 

0,151 

0.180 

0.181 

0.331 

0,378 

0.333 

0.539 

0,835 

0.535 

0.339 

0.335 

0.300 

0.371 

0.134 

0,147 

0.108 

0,191 

0.319 

0.330 

0.187 

0.538 

0.038 

0.031 

0.335 

0.393 

0.335 

0.383 

0.133 

0.107 

0,145 

0,309 

0,155 

0.337 

0,309 

0.593 

0,039 

0.017 

0.303 

0.388 

0.417 

0.438 

0,133 

0.135 

0,183 

0.170 

0.338 

0,305 

0.339 

0.583 

0,038 

0,039 

0.404 

0.388 

0.500 

0.513 

0,117 

0,137 

0,103 

0,179 

0,333 

0.359 

0,374 

0,093 

0.000 

0.013 

0.413 

0.413 

0.450 

0.475 

0.300 

0.109 

0,184 

0.304 

0,347 

0.390 

0.344 

0.036 

0.013 

0.035 

0.475 

0.379 

0.417 

0.479 

0,171 

0,177 

0.110 

0.188 

0,183 

0.331 

0.308 

1-8 


TABLE  1.6  CELL  MEANS  AND  VARIANCES 
SWAT 
(n  ■  4) 


With  Hauriatica  Without  Hauriatioa 

Station  Station 


l 

3 

3 

4 

i 

2 

3 

4 

Color 

Faat 

mtan 

47.3 

61.3 

44.8 

63.8 

46.3 

37.2 

51.7 

atd 

3.1 

3.6 

8.2 

8.6 

13.0 

8.0 

10.4 

Mod 

mtan 

34.1 

36.2 

20. 1 

36.8 

31.4 

34.4 

23.7 

33.1 

atd 

7.6 

3.2 

8.2 

6.1 

20,8 

11.3 

11.7 

30.4 

Alpha 

Faat 

mtan 

ESI 

42. 1 

48.1 

48,8 

40.8 

35.3 

42.0 

atd 

Bn 

8.7 

13,6 

0.8 

11.4 

5.4 

10.7 

Mod 

mtan 

30.8 

36.8 

20.7 

23.3 

33.3 

32.1 

17.5 

26.7 

atd 

0.7 

8.0 

5.8 

8,7 

10.  1 

22,2 

14.0 

20,0 

CuitJ 

Faat 

mtan 

63.3 

42.2 

49.4 

44.0 

60,3 

atd 

11,4 

8.1 

7.3 

9.0 

Mod 

mtan 

E 

40,4 

30,6 

Exj 

34.4 

41.9 

29.0 

34.4 

atd 

D 

31,5 

19,4 

mi 

P.3 

13.0 

20.0 

10. 8 

I- 10 


TABLE  I . 10  Raw  Data 
Alphabetic  prasantation  form,  with  hauriitioi 


Toam  1  Fast  Salts  1 

2 


Mod  Sail  1 
2 

3 

4 


Team  2  Fast  Sals  1 


Mod  Sail  1 
2 

3 

4 


Team  3  Fast  Sail  1 


Mod  Sail  1 
2 

3 

4 


Team  4  Fait  Sail  1 


Mod  San  1 
2 

3 

4 


1PL-1PT 

2PL-2PT 

2PL-4PT 

3PL-BPT 

XCERT 

SCORE 

SWAT 

0.040 

0,000 

0.830 

1.000 

0.682 

63.8 

38.2 

0.000 

0,000 

0.870 

0.880 

0.869 

01.7 

84.8 

0.040 

0.040 

0,710 

0.880 

0.867 

70.2 

47.0 

0,040 

0.000 

0,480 

1,000 

0,464 

69.2 

88,9 

0,040 

0.080 

0.780 

1,000 

0.616 

79.2 

30.0 

0,130 

0,040 

0,7d0 

0,880 

0.861 

73,3 

32.8 

0.080 

0.040 

0.780 

0.880 

0.884 

81.7 

17.3 

0.170 

0,000 

0,800 

0.780 

0.409 

77.2 

27,9 

0,000 

0,040 

0,460 

1.000 

0.802 

70.1 

41,2 

0,080 

0,000 

0.710 

1.000 

0.637 

71,1 

33,8 

0.170 

0,000 

0.880 

1.000 

0.809 

73.6 

40.6 

0,080 

0,000 

0.840 

0.880 

0.874 

09,2 

41,8 

0,080 

0.040 

0.710 

1.000 

0.828 

82.8 

18,9 

0,000 

0,000 

0.710 

1,000 

0,669 

78,8 

20,3 

0,170 

0.000 

0,790 

1.000 

0.882 

74.4 

13.8 

0.080 

0,000 

0,710 

1.000 

0,813 

89.6 

13,3 

0.000 

0.000 

0.130 

1.000 

0.338 

63.3 

22.8 

0,040 

0.000 

0.130 

0.830 

0.349 

38.1 

33.7 

0.000 

0.000 

0.130 

1.000 

0.288 

73.4 

29.2 

0.000 

0,000 

0.130 

0.880 

0.331 

68.3 

32.1 

0  .  (Mu 

0,000 

0.210 

0.880 

0.401 

71.3 

21.8 

0 ,  u  j(J 

0.000 

0,200 

1.000 

0.297 

83.4 

16.7 

0.040 

0.040 

0.210 

1.000 

0.416 

80.4 

20.8 

0.040 

0.000 

0.840 

1.000 

0,464 

82.4 

27,  1 

0,040 

0,000 

0.  130 

1.000 

0.889 

63.4 

40.8 

0,040 

0.000 

0,460 

1.000 

0.693 

06,0 

63.0 

0.040 

0.000 

0.840 

1.000 

0.834 

76.6 

31.6 

0.040 

0,000 

0.330 

1.000 

0,484 

67.8 

89.3 

0.000 

0,000 

0.880 

1,000 

0.838 

03.6 

18.7 

0,130 

0.000 

0.480 

1.000 

0.868 

83.7 

33.3 

0.040 

0.040 

0.330 

0.880 

0.481 

81.2 

28.8 

0.080 

0.040 

0,420 

1.000 

0,480 

84.8 

22,7 

TABLE  ! .  10,  Continued 

Alphabetic  presentation  form,  without  heuristics 


1FL-1PT 

2PL-2PT 

2PL-4PT 

3PL-5PT 

?{CERT 

SCORE 

SWAT 

Team 

5 

Fas  t 

Seas 

1 

0.000 

0.000 

0.130 

1.000 

0.402 

55.4 

51.2 

2 

0.040 

0,000 

0.250 

1.000 

0.545 

50.0 

43.0 

3 

0.000 

0.040 

0.040 

1.000 

0.412 

68.3 

30.0 

4 

0.000 

0.000 

0.040 

1.000 

0.560 

51.8 

33.7 

Mod 

Sess 

T 

0.000 

0.000 

0.200 

1.000 

0.501 

67.2 

20.5 

2 

0.080 

0.000 

0.420 

1.000 

0.503 

73.0 

14.7 

3 

0.040 

0.000 

0.210 

1.000 

0.442 

72.0 

5.6 

4 

0.000 

0.000 

0.130 

1.000 

0.443 

73.5 

15.6 

Team 

0 

Fast 

Sees 

1 

0.040 

0.080 

0.420 

1.000 

0.827 

60.8 

56.5 

2 

0.080 

0.000 

0.540 

1.000 

0.820 

66.8 

36.5 

3 

0.040 

0.000 

0.540 

1.000 

0.803 

68.2 

31.4 

4 

0.040 

0.130 

0.580 

0.680 

0.775 

71.0 

31.4 

Mod 

Sess 

1 

0. 170 

0.000 

0.630 

0.880 

0.687 

72.2 

2d,  4 

2 

0.080 

0.700 

0.750 

1.000 

0.751 

70.0 

28.8 

3 

0.130 

0.000 

0.540 

1.000 

0.726 

75.3 

20,2 

4 

0.130 

0.000 

0.530 

1,000 

0.782 

60.8 

22.4 

Team 

7 

Fast 

Sess 

1 

0.080 

0. 170 

0.450 

1.000 

0.737 

71.7 

51.3 

2 

0.080 

0.040 

0.580 

1.000 

0.732 

77.2 

83.5 

3 

0.080 

0.000 

0.540 

1.000 

0.602 

68.3 

30.1 

4 

0.170 

0.040 

0.630 

1,000 

0.672 

74.5 

66.0 

Mod 

Sess 

1 

0.000 

0.040 

0.250 

0.750 

0.400 

90.8 

48.3 

2 

0.210 

0.000 

0.580 

0.800 

0.603 

70.1 

64.2 

3 

0.040 

0.000 

0.420 

0.880 

0 .  -H-. 

04.0 

36.3 

4 

0.080 

0.000 

0.420 

1.000 

0  •  D  4^ 

05.6 

55.4 

Team 

8 

Fas  t 

Sess 

1 

0.040 

0.000 

0.000 

0.750 

0.252 

64.0 

40.3 

2 

0.000 

0.000 

0.000 

0.680 

0.301 

55.1 

41.1 

3 

0.000 

0.000 

0.000 

0.500 

0.153 

63.3 

40.6 

4 

0.040 

0.000 

0.040 

0,630 

0.205 

67.0 

35,7 

Sess  1 

o.oco 

0.000 

0.210 

1.000 

0.332 

78.7 

27.0 

2 

0.000 

0.000 

0.130 

0.880 

0.284 

77.8 

20.0 

3 

0.000 

0.080 

0.080 

0.500 

0.203 

82.7 

8.1 

4 

0.000 

0.000 

0.080 

0.360 

0.170 

79.8 

9.2 

Mod 


T All IjE  1.10,  Continued 

CMor  presentation  form,  with  heuristics 


1PL-1PT 

2PL-2PT 

2 PL- 4 FT 

3PL-8PT 

XCERT 

SCORE 

SWAT 

T?nm 

9 

Fast 

Sess  1 

0.040 

0.000 

0.210 

0.780 

0.277 

77.8 

44.0 

2 

0.040 

0.040 

0.290 

0.800 

0.288 

86.3 

48.8 

3 

0.000 

0.000 

0.210 

1.000 

0.348 

71.8 

88.3 

4 

0.000 

0,000 

0.170 

0,780 

0,289 

08.2 

38.8 

Mod 

Sess  1 

0.040 

0.040 

0.330 

0.780 

0.291 

90.3 

26.4 

2 

0.040 

0,000 

0.290 

0,880 

0.336 

81.8 

37,3 

3 

0.040 

0.000 

0.130 

1.000 

0.290 

91.8 

30.0 

4 

0.040 

0.040 

0.290 

0.880 

0.371 

91.8 

40, 1 

'loam 

10 

Fast 

Sard  1 

0.040 

0.000 

0.420 

1 .000 

0.460 

67.6 

60.9 

2 

0.000 

0.000 

0.290 

1.000 

0.411 

73.3 

47.3 

3 

0.040 

0.000 

0.380 

1.000 

0.386 

74.4 

40.0 

4 

0.000 

0.000 

0.280 

1.000 

0.389 

77.6 

46.3 

Mod 

Sess  1 

0.040 

0.000 

0.330 

1.000 

0.489 

81.2 

39.4 

2 

0.040 

0.000 

0.800 

1.000 

0.417 

86.6 

38.8 

3 

0.040 

0,000 

0.840 

1.000 

0,808 

92.2 

24.2 

4 

0.000 

0.000 

0.420 

1.000 

0.388 

93 » 8 

27.3 

Team 

11 

Fast 

Sees  1 

0.000 

0.000 

0.290 

1.000 

0.490 

58.6 

48.8 

2 

0,000 

0.000 

0.280 

1.000 

0,301 

69.1 

88,9 

3 

0.040 

0.000 

0. 130 

0.880 

0.240 

70.3 

36.9 

4 

0.000 

0.000 

0,080 

0.880 

0.234 

80.5 

06.4 

Mod 

Sess  1 

0,000 

0.000 

0.420 

1.000 

0.391 

74.4 

41.7 

2 

0.040 

0.000 

0.460 

1.000 

0.385 

82.8 

34.5 

3 

0.000 

0.000 

0.200 

1.000 

0.354 

81.9 

20.9 

4 

0.000 

0.040 

0.420 

0.780 

0.314 

82.9 

40.2 

Team 

13 

Fast 

Sess  1 

0.000 

0.000 

0 . 080 

0.880 

0.348 

62.3 

45.3 

2 

0.000 

0.000 

0.080 

0.780 

0.273 

72.2 

53.0 

3 

0.040 

0.000 

0.130 

1.000 

0.328 

70.5 

46.6 

4 

0.000 

0.000 

0.290 

1.000 

0.367 

00.4 

47.5 

Mod 

Sess  1 

0.040 

0.000 

0.130 

0.880 

0.324 

82.2 

28.9 

2 

0.000 

0.000 

0.210 

0.780 

0.282 

80.9 

42.2 

3 

0.000 

0.000 

0.080 

0.800 

0.228 

88.6 

32,8 

4 

0.000 

0.000 

0.290 

0.780 

0.309 

89.3 

34.3 

1-13 


TABLE  I. 10,  Continued 

Color  presentation  form,  without  heuristics 


1  PL-  PT 

3PL-3PT 

2PL-4PT 

3FL-5PT 

XCERT 

SCORE 

SWAT 

Team 

13 

Fast 

Sess  1 

0.040 

0.080 

0,540 

0.880 

0.586 

72.5 

28.4 

3 

0 .080 

0.310 

0.880 

1.000 

0.850 

72.7 

38,5 

3 

0.080 

0.000 

0.630 

1.000 

0.741 

66.1 

36. 1 

4 

0.080 

0.040 

0.630 

0,880 

0.672 

71.6 

30.5 

Mod 

Sess  1 

0.040 

0.460 

1,000 

0.431 

81.7 

12.3 

3 

0.170 

0,170 

0.710 

0.500 

0.728 

71.8 

30.6 

3 

0.130 

0.130 

0.710 

1.000 

0.767 

71.6 

14.6 

4 

0.310 

0.170 

0.540 

0.030 

0.642 

73.3 

22.8 

Team 

14 

Fast 

Sess  1 

0.350 

0.000 

0.670 

1,000 

0.003 

56.5 

34.7 

a 

0.080 

0.080 

0,580 

1.000 

0.880 

63.2 

32.1 

3 

0.380 

0.080 

0.570 

1.000 

0.087 

63.2 

26.3 

4 

0.170 

0,000 

0.350 

0.880 

0,666 

65.4 

54,4 

Mod 

Sess  1 

o.oeo 

0.000 

0.630 

1.000 

0.850 

69.6 

14.6 

3 

0,360 

0,080 

0.830 

1.000 

0,072 

73.1 

30.6 

3 

0.360 

0,310 

0.780 

0,880 

0.016 

70.5 

18.7 

4 

0.130 

O'.  040 

0.710 

1.000 

0.041 

7C.3 

13.0 

Teem 

15 

Fast 

Sees  1 

0.040 

0,000 

0.430 

1.000 

0.557 

65.6 

57.3 

7 

0.000 

0,000 

0.350 

1.000 

0.463 

71.8 

51.4 

3 

0.000 

0.000 

0.280 

1.000 

0.437 

73.8 

30.5 

4 

0.000 

0.000 

0.350 

1.000 

0.380 

73.0 

48.5 

Mod 

Sess  1 

0.310 

0.000 

0.330 

1.000 

0.508 

75.0 

50.6 

3 

0.130 

0.000 

0,310 

1.000 

0,444 

70.4 

44.0 

3 

0.000 

0.000 

0.310 

1.000 

0.370 

05.3 

33. 1 

4 

0.040 

0.000 

0.280 

1.000 

0.382 

04.2 

35,0 

Team 

16 

Fast 

Sess  1 

0.130 

0.040 

0.500 

1.000 

0.637 

50.2 

70.7 

> 

2 

0.000 

0.040 

0,630 

0.880 

0.840 

50.5 

03.4 

3 

0.040 

0,000 

0.500 

1.000 

0.664 

68.5 

47.0 

4 

0.130 

0.040 

0.330 

0.750 

0.686 

60.3 

64.3 

Mod 

Sess  1 

0.130 

0.130 

0.670 

0.750 

0.646 

68.0 

48.0 

3 

0.130 

0.130 

0.500 

1.000 

0.666 

74.2 

43.3 

3 

0.130 

0.040 

0.430 

0.880 

0.476 

84.5 

40,5 

4 

0.080 

0.080 

0.380 

0.750 

0.404 

80,0 

60.0 
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TABLE  I. 10,  Continued 

Conjunctiva  preaantat ion  form,  with  heurlatioa 


1FL-1PT 

2PL-2PT 

2PL-4PT 

3PL-BPT 

XCERT 

SCORE 

SWAT 

Team 

17 

Fast 

Sana  1 

0.000 

0.000 

0.080 

1.000 

0.288 

71.1 

51.8 

2 

0.000 

0.000 

0 .  ?B0 

1.000 

0,367 

71,8 

80,8 

3 

0.000 

0.000 

0.170 

1.000 

0.382 

71,6 

30.4 

4 

0.000 

0.040 

0.130 

1.000 

0.342 

64.8 

55.2 

Mod 

San  1 

o.oeo 

0.130 

0.330 

1.000 

0.408 

83.6 

25.3 

2 

0.000 

0.040 

0.880 

1.000 

0.462 

85.6 

21.2 

3 

0.000 

0.000 

0,280 

1.000 

0.310 

91.5 

13.5 

4 

0,080 

0.040 

0,840 

1.000 

0.817 

86.4 

16.1 

T«nm 

18 

Fiat 

Soil  1 

0.000 

0.260 

0.880 

0.462 

83.0 

48.1 

2 

0,000 

0.380 

0,880 

0.477 

66,7 

38,5 

3 

0,040 

0.840 

1.000 

0.030 

65.4 

46.9 

4 

4 

ms 

0.000 

0.380 

1.000 

0.883 

70.9 

40.6 

Sail  1 

0.080 

0.420 

0.780 

0.428 

75.6 

44.1 

2 

KKTnB 

0.000 

0.330 

1,000 

0,448 

72,8 

25,1 

■ 

3 

0.000 

0.420 

1.000 

0.800 

76.7 

16.3 

1 

4 

fKWtTTS 

0.040 

0.420 

1.000 

0.458 

02.8 

30.6 

Taam 

19 

Faat 

Sasa  1 

0,040 

0.000 

0.130 

0.780 

0.274 

84.7 

43.0 

2 

0,000 

0.040 

0,420 

1.000 

0.387 

76.2 

51.0 

3 

0,000 

0.040 

0.330 

1.000 

0.468 

83.3 

41.5 

4 

0.040 

o.oeo 

0,480 

0,880 

0.493 

'2.5 

43.4 

_ 

Mod 

Sen  1 

0.000 

0.080 

0.280 

0.880 

0.333 

75.4 

38.3 

2 

0.000 

0.000 

0.800 

1.000 

0.463 

07.3 

47. 8 

3 

0.080 

0.040 

0.840 

1.000 

0.504 

01.0 

37.5 

4 

o.oeo 

0.000 

0,870 

1.000 

0.573 

82.5 

42.1 

T*)nm 

20 

Fait 

Sa««  1 

0.000 

0.000 

0.290 

1.000 

0.383 

71.7 

65.8 

2 

0.000 

0.000 

0,170 

1.000 

0.413 

09.2 

60.3 

3 

0.000 

0,000 

0,280 

1,000 

0.444 

63.9 

72.1 

4 

0.040 

0.000 

0.380 

0.780 

0,363 

69.1 

59.3 

Mod 

Sa«i  1 

0.000 

0.000 

0.170 

1.000 

0.322 

77.8 

46.3 

2 

0.000 

0.040 

0,420 

1.000 

0,404 

81,8 

87.5 

3 

0.000 

0,040 

0.330 

1.000 

0.361 

81.0 

54.8 

4 

0.040 

0,000 

0,480 

1 .000 

0,468 

04,4 

52.5 

TABLE  I. 10,  Continued 

Conjunctive  preaentation  form,  without  heuriatica 


1PL-1FT 

2PL-2PT 

2PL-4PT 

3PL-5PT 

XGEKT 

SCORE 

SWAT 

Team 

21 

Feat 

Seaa  1 

0.040 

0.000 

0.170 

0.380 

0.221 

50.9 

58.8 

2 

0.000 

0,000 

0.330 

0.500 

0.328 

02.2 

00.9 

3 

0.000 

0.000 

0.380 

0.780 

0.383 

09.1 

52.1 

4 

0,000 

0,040 

0.210 

1,000 

0.355 

73.2 

02.0 

Mod 

Seaa  1 

0.000 

0,000 

0.280 

o.eeo 

0.312 

74.2 

59.2 

2 

0.040 

0,130 

0.380 

0.750 

0.399 

71.1 

01.5 

3 

0,000 

0,040 

0,210 

1,000 

0.300 

82.0 

50.9 

4 

0.000 

0.000 

0.290 

1.000 

0.401 

78.3 

35.9 

Team 

22 

Faat 

Seaa  1 

0,040 

0.080 

0.540 

1.000 

0.714 

85.5 

41.1 

2 

0,000 

0,000 

0,420 

1,000 

0.008 

71,0 

40.9 

3 

0,210 

0,000 

0.800 

0.880 

0.010 

08.3 

35.2 

4 

0,040 

0.000 

0.330 

0.500 

0.553 

07,7 

41.0 

Mod 

Seaa  1 

0.040 

0.080 

0.480 

0.880 

0.850 

80.3 

25.0 

2 

0,040 

0.080 

0.380 

1.000 

0.881 

79.4 

39.0 

3 

0.080 

0.130 

0.420 

0.880 

0.003 

77.0 

9.4 

4 

0,080 

0,040 

0.400 

1,000 

0.520 

70.0 

18.8 

Team 

23 

Faat 

Seaa  1 

0.080 

0.3B0 

1.000 

0.805 

50.1 

36.5 

2 

0.040 

0.000 

0.330 

1,000 

0.400 

70.3 

48.1 

3 

0.000 

0.170 

0.750 

0.337 

03.4 

47.1 

4 

0,000 

0.170 

0.880 

0.202 

68.2 

32.9 

"MoT 

Seaa  1 

0.080 

0.450 

1.000 

0.480 

72.4 

32.4 

2 

0.040 

0.400 

1.000 

0.819 

70.0 

30.8 

3 

0.080 

0.130 

0.330 

0.880 

0.333 

78.9 

32.8 

4 

0,000 

0.000 

0,250 

1,000 

0.324 

70.0 

39,4 

Team 

24 

Faat 

Seaa  1 

0 . 080 

0.170 

0.710 

1.000 

0.B81 

04.1 

32.8 

2 

0,250 

0.210 

0,790 

1,000 

0.082 

07.4 

41.9 

3 

0.080 

0.130 

0.710 

0.880 

0.883 

03.9 

41.0 

4 

0.210 

0.210 

0.070 

0.880 

0.810 

63.0 

44.0 

Mod 

Seaa  1 

0.080 

0.130 

0.710 

1.000 

0.706 

78.9 

20.2 

2 

0.080 

0,130 

0.710 

0.880 

0.814 

09.1 

35.8 

3 

0,210 

0.080 

0.750 

1.000 

0.774 

09.7 

19.4 

4 

0.170 

0.830 

0,880 

0.743 

73.3 

23.5 
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TABLE  I. 11  Raw  Data,  Uncertain  Event* 
Alphabetic  preaentation  form,  with  hauristlca 


INFO/81 

INF0/S2 

INF0/S3 

1 

1121 

Faat 

0.300 

0.480 

0.400 

Mod 

0.400 

0.400 

0,480 

HO 

Faat 

WEM 

■soi 

0,800 

Mod 

BBS 

IBB 

0.600 

L36 

Faat 

0.300 

■fiZfl 

0.300 

Mod 

0.300 

RE9 

0.600 

L04 

Faat 

0.200 

0.380 

Mod 

0.480 

0.800 

2 

H21 

Faat 

0.400 

0,260 

Mod 

0.080 

EfU 

0.680 

HO 

Faat 

0.480 

0.480 

0.800 

Mod 

0.800 

0.400 

0.600 

L38 

Faat 

0.380 

0.460 

0.300 

Mod 

0.080 

0.380 

0.630 

L84 

Faat 

BB 

■sgl 

0.660 

Mod 

0,800 

3 

H21 

Faat 

0.700 

0.860 

0.800 

Mod 

0.780 

0.780 

0.800 

HO 

Faat 

0.800 

0.600 

0.380 

Mod 

0.800 

0.700 

0.700 

L30 

Faat 

0.700 

0.800 

0.480 

Mod 

0.800 

0.780 

0.600 

L84 

Faat 

0.480 

0.680 

0.680 

Mod 

0.880 

0.780 

0.680 

4 

1121 

Faat 

0.380 

EBB 

0 ,  BOO 

Mod 

0.880 

WEM 

0.880 

HO 

Fait 

1 

0.480 

0.880 

0.380 

Mod 

0.380 

0.680 

0.030 

Faat 

0.880 

0.380 

0.630 

Mod 

0.300 

0.800 

0.700 

L84 

Faat 

0.380 

0.400 

Mod 

0.880 

PUB 

0.880 

1NF0/S4 

COMMIT 

0.880 

0.862 

0.330 

0,017 

QQ1 

0.000 

BBS 

0.000 

0.380 

0.000 

0.880 

0.000 

0.600 

BB 

0.780 

8KB 

0.400 

0.621 

0.600 

0.000 

0.330 

0.000 

0.280 

0.000 

0.880 

0.030 

0.400 

0.000 

0.400 

0.027 

0.000 

0.000 

0.600 

0.887 

0.430 

0.918 

0.780 

0.000 

0.700 

0.000 

0.800 

0.000 

0.630 

0 . 000 

0.600 

0,013 

0.700 

0,962 

0.680 

0.072 

0.700 

0.070 

0.780 

0.000 

0.480 

0.024 

0.880 

0,000 

0.880 

0.000 

0.380 

0,603 

0.600 

0,610 
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TADLE  1. 11,  Continu«d 
Alphibitic  pr««*nt»tion  form, 

INFO/51 


without  h«uri*tic« 

INF0/S2  INF0/S3  INFQ/S4  COMMIT 


Tmm  5 


Flit  0.400 


0 .  BOO 


0.680 


0.400 


0.680  0.800  0.780  0.780 


0,800  0.600 


0.800 


0.800 


0.880 

0.880 


0.600  0.480 


0.700 


l  .000 
1.000 


0.762 

0.786 


0.033 

0.000 


0.133 

0.061 


TABLE  1,11,  Continued 
Color  preaentation  form, 


INFO/S 


Team  9 

1121 

Feat 

0.700 

Mod 

0,900 

H6 

Feat 

0,700 

Mod 

0.600 

L36 

Feat 

0.600 

Mod 

0,750 

L84 

Feat 

0.660 

Mod 

0.750 

Teem  10 

1(2 1 

Feat 

HQ 

Mod 

Bffl 

118 

Feat 

0.400 

Mod 

0,550 

L30 

Feat 

0.550 

Mod 

0,700 

L04 

Feat 

0.550 

Mod 

0,750 

Team  1 1 

H21 

Feat 

0.500 

Mod 

0.650 

HO 

Feat 

0.750 

Mod 

0.050 

L30 

Feat 

0,050 

Mod 

0.750 

L84 

Feat 

0.050 

Mod 

0.750 

Teem  12 

1121 

Feat 

0.550 

Mod 

0.050 

HO 

Feat 

0.700 

Mod 

0.700 

LUO 

Feat 

0.050 

Mod 

0.950 

L84 

Feat 

0,700 

Mod 

0,800 

th  heuriatioa 
1  INFO/S2  INFO/S3 

0.700  0.700 

0.700  0,780 

0.560  0.550 

0.300  0.600 

0,800  0,750 

0.750  0,550 

0.650  0.550 

0.700  0.850 

0.600  0.600 

0.500  0.600 

0,400  0.450 

0.650  0.000 

0,450  0.450 

0,650  0.500 

0.450  0.650 

0,800  0.500 

0.700  0.650 

0.750  0.800 

0.050  oTooo 

0.700  0,700 

0.400  0.600 

0.800  0,800 

0.050  0.700 

0.700  0.550 

0.800  0.600 

0.050  0.800 

0,900  0.600 

0.000  0,850 

0,550  oTflOO 

0.800  0,000 

0.050  0.750 

0,900  0,850 


INF0/S4 

0.650 

0.550 

0.750 

0.850 

0.650 

0.700 

0.750 

0.850 

0.450 

0.700 

0.450 

0,850 

0.450 

0.450 

0.800 

0.000 

0,850 

0.800 

0.450 

0.700 

0.550 

0.800 

0.750 

0.550 

0,750 

0.850 

0,650 

0,750 

0,600 

0.850 

0.750 

0.700 


COMMIT 
0.843 
0  ■  883 
0,000 
0.000 
0.000 
0.000 
0.842 
0.908 
1.000 
0.977 
0.000 
0.000 
0.000 
0.045 
0,838 
1.000 
0.852 
0.882 
0.000 
0.000 
0.000 
0.000 
0.636 
0.082 
0.888 
1.000 
0.000 
0.000 
0 ,000 
0.015 
0.789 
0.860 
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TABLE  1,11,  Continued 

Color  prssintation  form,  without  haurisblcs 


INFO/SI 

2NFO/S2 

INFQ/S3 

INFO/S4 

COMMIT 

Tsam  13 

M2 1 

Fast 

0.480 

0.100 

0.180 

0.300 

0,050 

Mod 

0.480 

0.200 

0.2B0 

0.350 

0,080 

H9 

Fast 

0.280 

0.080 

0.200 

0.380 

Mod 

0.3B0 

0.180 

0,280 

0.200 

L36 

Fast 

0.280 

0.100 

0.180 

0,180 

0,385 

Mod 

0.400 

0.280 

0.300 

0,350 

0,833 

L84 

Fast 

0.200 

0.180 

0.280 

0.680 

Mod 

0.800 

0.280 

0,180 

0,618 

Tuam  14 

1121 

Fast 

0.000 

0.000 

0.000 

raia 

1.000 

Mod 

0.080 

0.000 

0.180 

1.000 

HO 

Fast 

ran 

0.000 

0,080 

0,400 

0.733 

Mod 

MEM 

0.080 

0.000 

0.000 

0.500 

L30 

Fast 

0.000 

0.080 

0.000 

0.300 

1.000 

Mod 

0. 100 

0.000 

0.080 

0.050 

0.500 

L84 

Fast 

0.080 

0.000 

0.000 

0.250 

0.887 

Mod 

0.180 

0.100 

0.080 

0.200 

0.400 

Tuam  IS 

H21 

Fast 

rang 

0.700 

0.880 

0.880 

0.020 

Mod 

| 

0.800 

0.780 

0.750 

0.042 

HO 

Fast 

0.480 

0.380 

0.480 

0.800 

0.027 

Mod 

0.880 

0,280 

0.700 

0, 580 

0.024 

L30 

Fast 

0.380 

0.860 

0.800 

0.880 

0.000 

Mod 

0.700 

0.780 

0.580 

0.800 

0.000 

LB4 

Fast 

0.800 

0,400 

0,800 

0.850 

0.684 

Mod 

0.800 

0.800 

0.880 

0.700 

0.030 

Tuani  10 

H21 

Fast 

0.400 

0.200 

0.480 

0.300 

0.083 

Mod 

0.800 

0.580 

0.550 

0.750 

0.038 

HO 

Fast 

0.400 

0.300 

0.180 

0.300 

0.565 

Mod 

0.380 

0.350 

0.400 

0.500 

0.408 

LUO 

Fast 

0.480 

0.300 

0.300 

rasa 

Mod 

0.480 

0.400 

0.480 

■IttiJi 

L84 

Fast 

0.400 

0.200 

0.300 

Mod 

0.800 

0.500 

0.550 

BuSI 
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TABLE  1.11,  Continue 
Conjunctive  present* 


Teem  17 

H21 

Feet 

Mod 

HO 

Feet 

Mod 

L38 

Felt 

Mod 

L84 

Felt 

Mod 

Teem  16 

H21 

Felt 

Mod 

HO 

Felt 

Mod 

136 

Felt 

Mod 

1.84 

Felt 

Mod 

Teem  10 

K21 

Felt 

Mod 

HO 

Felt 

Mod 

L36 

Felt 

Mod 

L64 

Felt 

Mod 

Tonm  20 

H21 

Felt 

Mod 

HO 

Felt 

Mod 

L38 

Feit 

Mod 

or 

Felt 

Mod 

4 

tlon  form,  with  heuristic* 


1NF0/S1 

INFO/83 

1HFO/S3 

INF0/S4 

COMMIT 

0.700 

0.680 

0.600 

0.750 

0.643 

0,690 

0.600 

0.700 

0.600 

0,061 

0.700 

0,660 

0.660 

0.750 

0,000 

0.700 

0.660 

0.500 

0,750 

0,000 

0.700 

0.600 

0.700 

0.000 

0,780 

0.660 

0.660 

0,000 

0.600 

0.600 

0.500 

0,600 

0.870 

0.600 

0.600 

0,850 

0.800 

0,080 

0.490 

0.500 

0.400 

0.380 

0.883 

0,680 

0.600 

0,600 

0.000 

0.057 

0,680 

0.350 

0.350 

0.350 

0,600 

0,600 

0,780 

0.500 

0.400 

0.600 

0.400 

0.400 

0.000 

0.800 

0.600 

0.400 

0.550 

0.000 

0,460 

0.450 

0.380 

Dll 

0.941 

0.600 

0.660 

0.450 

Emu 

0.881 

0.600 

0.660 

0.460 

0.350 

0.002 

0.750 

0,660 

0.680 

0.400 

0.970 

0.650 

0.400 

0.350 

0.400 

0.000 

0.060 

0.360 

0.450 

0.450 

0.000 

0.650 

0.400 

0.600 

0.400 

0.024 

0.800 

0,660 

0.500 

0.400 

0.000 

0.600 

0,800 

0.400 

0.880 

1.000 

0.600 

0,600 

0,400 

0.500 

0.976 

0,960 

0.400 

0.800 

0.450 

0.947 

0.660 

0,950 

0,700 

0,550 

0.959 

0.680 

0.880 

0.550 

0.850 

0.000 

0,600 

0,600 

0.750 

0.600 

0.020 

0.300 

0.750 

0.800 

0,600 

0,000 

0.700 

0.300 

0.650 

0.700 

0.000 

0.850 

0.650 

0.600 

0.750 

0.863 

0.700 

0.800 

0.600 

0.650 

0,964 
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TAME  1.11,  Continued 

Conjunctive  presentation  form,  without  heuristics 


INFO/S  1 

1NF0/B2 

INFO/S3 

1NFO/S4 

COMMIT 

IT  earn  21 

K21 

Fast 

0.400 

0.400 

0.400 

wxnm 

1,000 

Mod 

0,450 

0.400 

0.550 

0.974 

H9 

Fast 

0.500 

0.400 

0.950 

B 

0.959 

Mod 

0.400 

0.450 

0.450 

n 

0.704 

L39 

Fast 

0.450 

0,500 

0.250 

0,300 

0.933 

Mod 

0,950 

0,500 

0.250 

0,300 

1.000 

L3  4 

Fast 

0.400 

0.400 

0.450 

m 

1,000 

Mod 

0,450 

0.400 

0.450 

0,970 

Team  22 

H21 

Fast 

0.200 

■ZB 

0.900 

0.500 

0.900 

Mod 

0.550 

IBM 

0.900 

0.400 

0,917 

K9 

Fast 

0,200 

0.250 

0.490 

0.000 

Mod 

0.200 

0,050 

0,900 

0,000 

L39 

Fast 

0,400 

0.450 

0.290 

0.990 

0,000 

Mod 

0.400 

0,700 

0.700 

0.500 

0.000 

L84 

Fast 

0.300 

0,490 

n 

0.300 

0.990 

Mod 

0.450 

0.500 

HTTHi 

0.050 

0.897 

Teem  23 

H21 

Fast 

0.290 

0.050 

0.700 

0.950 

0.897 

Mod 

0,990 

0.900 

0.750 

0,700 

1.000 

H9 

Fast 

0.390 

0.950 

0.400 

0.400 

Mod 

0.490 

0.900 

0.900 

0.800 

EE5 1 

L39 

Fast 

0.490 

0.450 

0.590 

0.550 

0.029 

Mod 

0,250 

0.450 

0.850 

0.950 

0.  114 

L84 

Fast 

0.400 

0.550 

0.750 

0.907 

Mod 

0.500 

0,500 

0.900 

0.790 

1'eem  24 

H21 

Fast 

0.050 

0.100 

0.050 

0.050 

1.000 

Mod 

0,290 

0,090 

0.100 

0.300 

0.057 

HO 

Fast 

0.100 

o .  ac 

0.200 

0.000 

Mod 

0.200  ' 

0.300 

0.200 

0.700 

L39 

Fast 

0.090 

0,000 

0.050 

0.200 

0.933 

Mod 

0.190 

0.100 

0.190 

0. 100 

0.600 

L84 

Fast 

0.100 

0.190 

0.050 

0.150 

0.778 

Mod 

0.200 

0.200 

0.150 

0.200 

0,933 
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